An Evaluation of Behaviour Oriented Design for Robotic Applications
126
An Evaluation of Behaviour Oriented Design for Robotic Applications Jason Llewellyn Leake MSc in Advanced Computer Systems: Internet Systems 2010
Transcript of An Evaluation of Behaviour Oriented Design for Robotic Applications
An Evaluation of Behaviour Oriented Design for Robotic Applications
Jason Llewellyn Leake
MSc in Advanced Computer Systems:
Internet Systems 2010
An Evaluation of Behaviour Oriented Design for Robotic Applications
submitted by Jason Llewellyn Leake
for the degree of MSc
of the University of Bath
COPYRIGHT
Attention is drawn to the fact that copyright of this thesis rests with its author. This copy of the thesis
has been supplied on condition that anyone who consults it isunderstood to recognise that its copyright
rests with its author and that no quotation from the thesis and no information derived from it may be
published without the prior written consent of the author.
Declaration
This dissertation is submitted to the University of Bath in accordance with the requirements of the
degree of Master of Science in the Department of Computer Science. No portion of the work in this
thesis has been submitted in support of an application for any other degree or qualification of this or
any other university or institution of learning. Except where specifically acknowledged, it is the work
of the author.
Summary
Despite its early development being largely focused on robotics, Behaviour Oriented Design has lan-
guished in this problem domain for a decade, whilst it has been used successfully in a range of others.
Meanwhile robotics has developed with ever more sophisticated architectures and hardware. This work
tests the thesis that Behaviour Oriented Design is an appropriate and useful technique, either in its cur-
rent form or with some modification, for implementing reactive planning in a modern robot architecture.
It uses BOD to implement a robotic application and then makesa critical examination of the problems
and benefits encountered using the methodology.
1
AcknowledgementsI should like to acknowledge the following people:
• My supervisor Joanna Bryson for her helpful discussions andguidance. She created Behaviour
Oriented Design, so I literally could not have undertaken this project without her.
• Tony Belpaeme, Paul Fitzpatrick, Giorgio Metta, Anthony Morse, Martin Proetzsch and Vadim
Tikhanoff, who have all provided useful discussions, explanations and advice on various aspects
of robotics and artificial intelligence.
• Andrew Lee and Dave Harbourn for agreeing to be my “naive users”, Ben Binsted for his com-
ments and discussion, and Christine Leake for her proof-reading.
2
Contents
Acknowledgements 2
1 Introduction 8
1.1 Deliberative and reactive planning . . . . . . . . . . . . . . . . .. . . . . . . . . . . 8
1.2 Behaviour Oriented Design . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 9
1.3 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 9
2 Background 11
2.1 Action selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 11
2.2 Reactive planning in robots . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . 12
2.3 Behaviour Based AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 12
2.4 Behaviour Oriented Design . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 13
2.5 BOD tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 15
2.6 Existing applications using BOD . . . . . . . . . . . . . . . . . . . .. . . . . . . . . 15
2.6.1 Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.6.2 Animal behaviour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 16
2.6.3 Non-player characters in computer games . . . . . . . . . . .. . . . . . . . . 17
3 Background to the work undertaken 19
3.1 Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 19
3.2 The iCub robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 19
3.3 Robot middleware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 21
3.4 YARP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.5 Build environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 22
3.6 Simulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 23
4 Work undertaken 25
4.1 Project plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 25
4.2 Initial evaluation task . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . 26
4.3 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 28
4.3.1 POSH engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.3.2 POSH plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3
4.3.3 Behaviours . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
4.3.4 Interface between Python and control classes . . . . . . .. . . . . . . . . . . 28
4.3.5 iCub control classes . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 30
4.3.6 Coordinate systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 34
4.3.7 Coping with errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 34
4.4 Using the BOD Methodology . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 35
4.4.1 High level specification . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 35
4.4.2 Generate likely sequence of actions . . . . . . . . . . . . . . .. . . . . . . . 35
4.4.3 Generate a list of sense and action primitives . . . . . . .. . . . . . . . . . . 36
4.4.4 Identify the state necessary to enable the described primitives and drives . . . . 36
4.4.5 Prioritise the list of goals . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 37
4.4.6 Iterative Development . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 38
4.5 Vision processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 41
4.5.1 Lightweight vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 41
4.5.2 Visually guided grasping . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 41
4.5.3 Object location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 43
4.5.4 Hand location and movement . . . . . . . . . . . . . . . . . . . . . . .. . . 43
4.5.5 Initial algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 44
4.5.6 Replacement algorithm . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 45
4.6 Inverse kinematics for limb movement . . . . . . . . . . . . . . . .. . . . . . . . . . 46
4.6.1 What is Inverse Kinematics? . . . . . . . . . . . . . . . . . . . . . . .. . . . 47
4.6.2 KDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.6.3 Other approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 48
4.6.4 iKin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.7 Avoiding obstacles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 50
4.8 Grasping and picking up the object . . . . . . . . . . . . . . . . . . .. . . . . . . . . 51
4.8.1 Thumb/index finger pincer . . . . . . . . . . . . . . . . . . . . . . . .. . . . 52
4.8.2 A cube instead of a ball . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 52
4.8.3 Emulating the robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 52
4.8.4 Thumb/little fingers pincer . . . . . . . . . . . . . . . . . . . . . .. . . . . . 52
4.8.5 Simulator “grab” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 53
4.8.6 Other approaches to grasping . . . . . . . . . . . . . . . . . . . . .. . . . . . 54
4.9 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 54
4.9.1 Obsessive behaviour . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 55
4.10 Extending the original task . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . 56
4.10.1 Modifications to C++ control classes . . . . . . . . . . . . . .. . . . . . . . . 56
4.10.2 Behaviour modifications . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 56
4.11 Description of the behaviours . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . 58
4.11.1 FindingTable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 58
4.11.2 LocatingCube . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 59
4.11.3 PickingUpCube . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 59
4
4.11.4 RightHandGrasping . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 60
4.11.5 RemovingYellowCube . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 61
4.11.6 HandlingErrors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 62
4.12 Naive users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 63
4.12.1 Task design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 63
4.12.2 First user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 64
4.12.3 Second user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 64
5 Results 66
5.1 Evaluation of the BOD methodology . . . . . . . . . . . . . . . . . . .. . . . . . . . 66
5.2 POSH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.3 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 67
5.3.1 jyPOSH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.3.2 ABODE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.4 Naive user experiences . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . 68
5.5 Simulator problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 68
5.5.1 Poor fidelity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 68
5.5.2 Forces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.5.3 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.5.4 Simulated grasp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 69
6 Discussion and analysis of the results 70
6.1 BOD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6.2 Behaviours as objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . 71
6.2.1 Spotting opportunities and redundancy earlier . . . . .. . . . . . . . . . . . . 72
6.2.2 Interaction between behaviours . . . . . . . . . . . . . . . . . .. . . . . . . 73
6.3 BOD in software engineering . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 73
6.3.1 Requirements Analysis . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 73
6.3.2 Requirements Definition . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 73
6.3.3 System Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 74
6.3.4 Software Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 74
6.3.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 74
6.3.6 Unit Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 75
6.3.7 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 75
6.3.8 System Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 75
6.3.9 Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75
6.3.10 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 75
6.4 POSH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
6.5 Use of classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 76
6.5.1 Encouraging component reuse . . . . . . . . . . . . . . . . . . . . .. . . . . 77
6.6 Non-interruptible tasks . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . 77
5
6.6.1 A cleanup action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 78
6.6.2 Current goal allowed to continue . . . . . . . . . . . . . . . . . .. . . . . . . 78
6.7 Simultaneously active behaviours . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 78
6.7.1 Interaction between behaviours . . . . . . . . . . . . . . . . . .. . . . . . . 79
6.7.2 Split of functionality between behaviours . . . . . . . . .. . . . . . . . . . . 79
6.7.3 Relative priority ordering cannot be changed . . . . . . .. . . . . . . . . . . 80
6.8 Multiple goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 81
6.8.1 Non-conflicting goals . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 82
6.8.2 Conflicting goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 83
6.9 Abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 83
6.10 Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 83
6.11 State variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . 84
6.12 Multiprocessor systems . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . 84
6.12.1 POSH plans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .85
6.13 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 85
6.14 BRPs as part of something bigger . . . . . . . . . . . . . . . . . . . .. . . . . . . . 86
6.15 Test and verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 87
6.16 Expanding to a large system . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 88
6.17 How BOD should be extended . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 88
6.17.1 Extensions to the BOD methodology . . . . . . . . . . . . . . . .. . . . . . 88
6.17.2 Extensions to POSH . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 89
6.18 Non-BOD related issues . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . 91
6.18.1 Programming languages . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 91
6.19 Grasping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 92
7 Evaluation of the project 94
7.1 What has been achieved . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 94
7.2 Evaluation of the work . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 95
7.2.1 Was the best method used for evaluating BOD? . . . . . . . . .. . . . . . . . 95
7.2.2 The task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
7.2.3 jyPOSH engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .96
7.2.4 iCub robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
7.2.5 Real robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .97
7.2.6 Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
7.2.7 Other issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 97
7.3 How the project plan evolved . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 98
8 Conclusions 99
8.1 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 99
8.1.1 Extending the evaluation project . . . . . . . . . . . . . . . . .. . . . . . . . 99
8.1.2 Extending the BOD methodology . . . . . . . . . . . . . . . . . . . .. . . . 100
6
8.1.3 Areas other than BOD or its evaluation . . . . . . . . . . . . . .. . . . . . . 100
References 101
A Requirements 108
A.1 Requirements Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . 108
A.1.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
A.1.2 Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
A.1.3 POSH engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
A.1.4 Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
A.1.5 Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .109
A.2 Requirements specification . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . 110
B Using the software 111
B.1 Installing the software needed to run the simulated tasks . . . . . . . . . . . . . . . . 111
B.2 ODE 0.11.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 112
B.3 IPOPT 3.8.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 112
B.4 YARP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
B.5 Simfork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 113
B.6 Environment variables . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . 113
B.7 iCub code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 114
B.8 jyPOSH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .114
B.9 Running the software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 115
B.10 Ancillary code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 116
B.11 Subversion revisions . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 116
C Steps needed to port the code to a real iCub 117
D Differences between Simfork and official iCub simulator 119
E Robots versus simulators 121
E.1 Complexity of the simulator . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 122
E.2 Embodied cognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 123
E.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 123
F Contents of the CD 124
7
Chapter 1
Introduction
This chapter provides a brief introduction to Behaviour Oriented Design and the motivation for carrying
out the evaluation of its applicability to robotics.
1.1 Deliberative and reactive planning
The most important decision that an autonomous agent such asa robot needs to make is how to achieve
its goals. There are two main techniques. “Deliberative planning” builds an internal symbolic represen-
tation of the world around the agent and uses this model to derive an optimal sequence of actions that
will move the world from its current state to its desired goalstate.
A second technique is “reactive planning”, using the agent’s current state and inputs from its sensors
to select the next action it should perform to move towards its goals. It only looks one step in advance,
but once it has completed that action, the process is repeated to choose the next action. This produces
a simpler implementation because the complexities of planning are offloaded onto the human devel-
oper. Any deliberative planning system can, at least in principle, be implemented as a reactive planning
system.
The main advantages of reactive planning over deliberativeplanning are that it is fast to execute and
simplifies the design of the agent’s software since many reactive planning techniques use a hard-coded
set of rules determined in advance. It also allows the agent to operate in a chaotic and unpredictable
environment where deliberative planning cannot keep up with changes.
It is hard and time consuming to implement reactive planningfor very complex tasks, since the
programmer has to understand all of the conditions that the agent will encounter and form a view on
how to react to them. However it it can be extremely effectiveat implementing simple behaviours.
Consequently both approaches are usually required for the most sophisticated robot.
This is often done using a three layer “hybrid” model, with the bottom layer implementing reading
sensors and driving actuators, with a feedback loop betweenthe two. The middle layer is the reactive
planning layer, and controls the underlying layer. The uppermost layer is a deliberative planner which
handles the long term goals and interfaces with the middle, reactive planning layer by modifying its
goals (Gat, 1998).
8
An example is a robot which uses reactive planning to handle the mechanics of walking, but delib-
erative planning to decide where to walk to. Thus, the rapidly responding reactive plan deals with the
simple short term goals, whilst the slower deliberative plan looks after overall strategy.
1.2 Behaviour Oriented Design
One approach to reactive planning is Behaviour Oriented Design (Bryson & Stein, 2001). This is a
combined architecture and methodology that, at its most basic, represents each different action as an
object and uses fixed rules for action selection. It combinesthe lower two layers of the three layers into
a single layer which deals with both reactive planning and the lower level behaviours.
It was developed by Dr. Joanna Bryson in the late 1990s, when the focus was on the architecture,
at that time called Edmund. The problem initially considered was programming a mobile Nomad 200
robot to travel around a set of offices avoiding obstacles (Bryson & McGonigle, 1997).
Since that time BOD has been used in other domains, such as modelling the behaviour of animals
and controlling non-player characters in virtual reality games, but not in robotics.
1.3 Motivation
Behaviour Oriented Design is a mature methodology and architecture which has been used to implement
action selection in a range of domains.
Although the early development of BOD was in robotics it has never been reapplied in this area,
although it seems an obvious area and has been previously suggested (Kwong, 2003, page 60). Its lack
of later application to robotics is especially surprising because its model is based upon Behaviour Based
Artificial Intelligence, which originated in robotics (Brooks, 1991).
There are strong reasons for it being an attractive methodology:
1. Behaviour Based architectures are widely used in robotics
2. BOD has been applied to robotics, with the Nomad 200 work
3. It has been successful in a range of other problem domains
4. It is a software engineering methodology, providing a disciplined approach
5. It is both modular and straightforward
However there are also indications that it may not be successful:
1. The Nomad 200 robot used was very simple by modern standards
2. The Nomad 200 work only achieved three of the six original goals
3. Other techniques have been used with success, the authorsin many cases being aware of BOD
4. The available tools are relatively crude
9
5. Its application to other problem domains have thrown up minor problems
6. It has been criticised as being too high level, lacking rules for coordinating low level interactions
between behaviours and shared access to actuators (Proetzsch et al., 2007, 2010)
7. There are differences in the robotic problem domain compared to others
In his criticism of the Waterfall Model Royce (1970) pointedout that an elaborate approach is
needed for developing large systems. In evaluating the BOD methodology it is therefore of interest to
investigate how it might be used in the development of very large and complex robotic systems.
10
Chapter 2
Background
This chapter provides a description of the different strategies planning in robots and how BOD fits into
this. It then describes the other areas that BOD has been usedin, and gives a detailed account of the
BOD methodology and its action mechanism scheme, which is called POSH.
2.1 Action selection
Action selection, the choice of actions to achieve an autonomous agent’s goals, has been an area of
intense study both in artificial intelligence and biology. It can be viewed as a continuous search through
the space of possible actions looking for the optimum one.
There are many approaches to the problem – see Bryson (2007) for a brief review of the historical
development. The simplest is to carry out a predefined sequence of operations. This is only successful
if the environment is completely predictable and well understood at the time that the sequence of op-
erations is defined. In practice this may not be the case and there are usually several competing, and
possibly conflicting, goals. For example, an animal may simultaneously have goals of finding food,
finding water, finding a mate, detecting and avoiding predators, conserving energy and avoiding injury.
Some of these have a higher priority than others, and the priorities may depend upon the state of the
animal. For example the priority of finding water is high whenthe animal is very thirsty. Conflict can
easily arise, for example if the animal is very thirsty then it may look for water in areas where there is
an increased risk of encountering a predator.
The traditional artificial intelligence approach to actionselection is deliberative planning, a “hori-
zontal decomposition” of sense-plan-act, in which data from sensors is used to build a symbolic model
of the world. This model is used to select the optimum action or sequence of actions by searching
through the repertoire of possible actions and computing their outcomes. The selected action then gen-
erates outputs to actuators to implement that action.
If the environment is changing rapidly by the time that the next action has been selected the infor-
mation used to select it is already out of date. The environment may also be too complex to allow a plan
to be devised which will handle every circumstance and so it will require constant revision.
11
2.2 Reactive planning in robots
An approach which does not suffer from these problems is reactive (or dynamic) planning. The key idea
is to avoid long term strategy and use input from sensors to determine what the current goal should be,
and the immediate actions which need to be performed to get there.
Frustrated by the very slow speed of the sense-plan-act model which was conventionally used in
intelligent mobile robots in the 1980s Rodney Brooks (1986)implemented a reactive model which he
called “subsumption architecture”. This model featured layers built on top of each other, and in its
original form only implemented finite state machines. The bottommost layer (each layer is called a
“behaviour”) implements very simple functionality, just reading from sensors and driving actuators in a
simple way. In Brooks’ paper this layer merely prevented therobot from running into obstacles. Each
successive layer monitors some of the sensor inputs, and theoutputs of modules within the layers below
and drives the actuators. If it drives actuators then it overrides the underlying layer either by suppressing
its inputs or, more often, its outputs.
In this way each layer implements successively more complexbehaviour – in Brooks’ model, the
second layer implemented “wander aimlessly around”. The model is subsumptive because each layer
subsumes part of the functionality of the layers below into itself, performing a more complex version of
them.
The architecture proved very successful, because each layer is simple and easy to code (at least
whilst the interactions between modules in different layers remains simple), and because development
was incremental – each layer is developed and debugged before the next is added. The result is an
agent that can take advantage of any opportunity that may arise, and can adapt rapidly to any change in
circumstances.
Brooks (1991) describes how a robot designed to wander around the laboratory picking up soda cans
takes advantage of a can being handed it by a human, since the robot hand had a programmed reflex to
close when an object was placed in it (the arm and hand are described in Connell, 1989a).
Brooks and others used architectures based on this type of model to develop several robots (Brooks,
1990, describes nine). One particular feature he observed was that complex emergent behaviour could
be produced from a set of simple reflexes, for example a six legged robot capable of walking (Brooks,
1989).
2.3 Behaviour Based AI
Subsumption Architecture was the first to be described as Behaviour Based Artificial Intelligence, BBAI
(Brooks, 1991). It has proved extremely successful in robotics because of both its robustness and per-
formance (Maes, 1991b; Mataric, 1997). Many people have argued that this model is much closer to
the neuroscience of biological intelligence than the oldermodel of a central symbolic reasoning engine
(for example Brooks, 1990; Prescott et al., 1999).
Subsumption is by no means the only BBAI architecture – A widevariety of architectures have
followed it, such as Julio Rosenblatt’s DAMN (Rosenblatt, 1997) which has a central arbitrator selecting
the next action by counting the weighted votes for actions chosen by different goal-seeking modules,
12
and Jonathan Connell’s Colony Architecture (Connell, 1989b) with multiple microprocessors (24 in the
original implementation) and arbitration controlled by the physical connections between them so that
one module can suppress the output of a lower priority one.
One problem with behaviour-based, and other reactive, mechanisms is that they cannot plan longer
term strategy. They are very good at comparatively low leveltasks, in the case of robots such tasks as
travelling to a landmark but lack the capacity for longer term planning. One reason for this is that they
rely upon sets of rules which have been hard-wired into the agent; so the planner is unable to formulate
completely new sets of rules.
A common compromise (for example in Bonasso et al., 1997; Basu et al., 2008) is a hybrid multi-
layer architecture containing a reactive layer which looksafter short term goals and a deliberative layer
responsible for longer term ones. The classic three layer model is described by Gat (1998) – a planning
layer, or deliberator performs deliberative planning and its output is used to guide the goals of the
underlying sequencing layer which performs reactive planning. The sequencing layer in turn controls
a skill layer, which the implements low level behaviours andcan include tight feedback loops coupling
sensors and actuators together.
Thus in a walking robot, the planning layer may choose the destination for the robot, the sequencing
layer control walking by enabling or suppressing differentparts of the skill layer, and the skill layer
controls the movement of the joints.
There are other problems with behaviour-based artificial intelligence – it often requires a consid-
erable amount of hand-coding and getting the interaction between the different behaviours correct can
be very complicated since there are many possible interactions (the interaction between behaviours is
sometimes called “wires” in deference to Brook’s Subsumption architecture which used physical wires
even though now it is almost invariably happens within software). Learning can also be difficult to
implement since the intelligence is distributed across multiple quasi-independent behaviours, and so is
generally restricted to simple state information in each behaviour.
2.4 Behaviour Oriented Design
Behaviour Oriented Design (BOD) is a combination of a reactive planning architecture and a software
engineering design methodology. Bryson & Stein (2001) contains a concise description of it, for a
broader discussion see Bryson (2001a).
In part, the rationale for BOD was driven by the observation that automated planning and machine
learning are widely used in preference to hand coding BBAI systems because they were easier to imple-
ment, even though hand coding would be better in many cases. Bryson therefore put effort into devising
a methodology which was easier and more rigorous than ad-hochand coding.
The BOD architecture consists of:
• A library of behaviours, implemented as objects.
• A reactive plan, which in BOD is a POSH (Parallel-rooted, Ordered, Slip-stack, Hierarchical)
plan.
13
Each behaviour object represents a perception/action/learning module and implements the actions
needed to carry out its tasks. Behaviour primitives called from POSH are behaviour object methods
(in the object oriented programming sense), and are a combination ofactsor senses. There is actually
little difference between the two; an act can succeed or failwhilst a sense returns a value. Memory
(and learning) is implemented in the behaviour objects, usually as simple state variables since that is
generally all that is needed.
The processing of the reactive plan is absolutely key since it is this which decides when and un-
der what circumstances the various behaviours become active. The root of a POSH plan is thedrive
collection, a hierarchically arranged set of production rules for the agent’s goals and the triggers (also
called releasers) to activate them, along with the actions which need to be performed. On each cycle
of the planner the list is examined to find the highest priority goal which is triggered and the action
associated with it is performed. In the general case an action can be acompetence, which is again a list
of production rules – a list of predicates and actions associated with them. The list is traversed and each
predicate which is true causes its corresponding action to be performed. It can also be a list of actions
performed in sequence (called anaction pattern), or just a primitive act or sense. Every action hierarchy
must obviously terminate with acts and senses.
A competence element can contain four items:
1. A priority for checking the element, relative to other elements in the competence
2. A trigger condition which will cause the action to be performed
3. The action to be performed
4. An optional number of times to retry the action if it fails
An element in a drive collection can add a maximum frequency at which the trigger for the ele-
ment should be checked. This allows the agent to simulate a limited amount of parallel processing by
permitting other lower priority drives to be active when thehigher priority one does not need to be.
The POSH mechanism described above can be implemented as Scheduled POSH, which only allows
an action to operate for a limited amount of time before the drive collection is re-examined – at which
time the action either continues or is replaced by a new one. This allows an agent to interrupt an activity
to perform a higher priority one and the activity not be restarted if the higher priority one has taken
sufficiently long to make it not worthwhile (Bryson, 2001a, pages 72–74). An example discussed is
the Nomad 200 robot recovering from a collision with a wall asa high priority action which takes
sufficiently long to make resuming a sonar measurement not worthwhile. The non-scheduled version of
POSH is sometimes called SPOSH, for Strict POSH.
The BOD methodology emphasises iterative development cycles following the Spiral software de-
velopment model (Boehm, 1986). It is split into two parts: a decomposition of the problem in which the
primitives, behaviours and drives are identified, followedby iterative development of the reactive plans
and behaviours.
14
2.5 BOD tools
Bryson originally produced implementations of POSH using the Common Lisp Object System (CLOS),
Perl 5 and C++. A second CLOS implementation was written as part of a collaboration with LEGO.
Since then more tools have been added. The most important of these is the POSH implementation
written in Python by Andy Kwong (Kwong, 2003), a port from CLOS, which has been used in most
of the BOD projects since 2003. This has been modified to run under Jython (and renamed jyPOSH),
to allow it to interface to Java code easily so that it can be use the MASON multi-agent discrete-event
simulation environment.
Several POSH plan editors have been developed, but they all lack functionality, notably not being in-
tegrated with the jyPOSH engine to allow running plans to be monitored and debugged. Richards (2004)
carried out an undergraduate dissertation project to writean editor, “IDE|POSH”. as a replacement for
an earlier undergraduate project, PoshEdit. However the most successful editor is the Advanced Be-
haviour Oriented Design Environment, ABODE, described in Bryson et al. (2006) and Bryson (2010).
2.6 Existing applications using BOD
BOD has been applied to a variety of problem domains. The oneswhere the greatest amount of work
has been done are listed below, but this does not cover the full range. Bryson’s thesis suggested how
BOD could be used in several other areas, including modelling the action selection processes which
take place in the basal ganglia of mammals, and a computer-human dialogue system.
2.6.1 Robotics
Much of the early development of BOD took place using a Nomad 200 robot in the 1990s (Bryson,
2001a,b; Bryson & McGonigle, 1997, pages 109–115 of), and this remains the only reported robotics
application.
The Nomad 200 was a popular commercial research robot of the 1990s and is described in section
2 of Chopra et al. (2006). It was cylindrical device around 1 metre high and mounted on a three wheel
drive mechanism. The top section of the robot could rotate independently of the base, and was equipped
with 16 sonar sensors which each covered a small arc but combined to cover the full 360 degrees, and
complemented by 16 infrared sensors. Around the base of the robot were positioned two rings of 10
push sensors to sense collisions. The software ran under Unix on an Intel 80486 processor.
Six goals were originally set for the robot:
• Return to base if battery energy falls below threshold
• Set speed proportional to distance to nearest obstacle
• If robot finds itself in an area not previously mapped then mapit
• Remain near base if not busy
• Detect approaching people
15
• If a human waves their arms then follow the human until they stop moving
Only the first three goals were achieved, and the third was notreliable. The reasons for this were an
underestimate of the time needed for implementation and theselection of unsuitable mapping algorithms
given the very poor spacial resolution of the robot’s sonar system. Despite the partial success of the
project, Edmund was demonstrated to be a practical architecture.
Following this, some work was undertaken in implementing action selection on a simulated hu-
manoid robot to assist with the development of a neural network approach to grasping and vision.
Although this was not brought to a conclusion, it found bugs in the Nomad 200 implementation and so
identified untested functionality, notably that the robot experiment had lacked the complexity of several
difference behaviours interacting with each other.
Proetzsch et al. (2007) has criticised the use of BOD as a robot architecture, as the strictly prioritised
plans that BOD uses are appropriate for achieving the high level goals but are deficient in low level mo-
tion control. There are no heuristics for specifying the interaction between behaviours or for controlling
actuators Proetzsch et al. (2010).
The coordination between behaviours is limited to strict priority ordered action selection, allow-
ing only one corresponding to the highest priority goal to operate at any given time. This makes no
allowance for pursing several goals simultaneously, except by switching between behaviours, since it
does not permit several different behaviours needing to runat the same time. Since only one behaviour
can run at any time, different behaviours cannot work together and interact to provide a compromise be-
tween all of them. Pirjanian (2000) divides action selection mechanisms into “arbitration” types (which
includes BOD) that choose which action to perform, and “command fusion” types which coordinate si-
multaneously running behaviours. Rosenblatt’s DAMN as an example, since several behaviours operate
at the same time and the actual action performed is decided byvoting between them.∗
The authors of this paper Proetzsch et al. (2007) proposed another architecture, Integrated Behaviour-
Based Control (iB2C), which adopts BOD’s approach in using amethodology (or at least guidelines)
and provides tools. The architecture however is somewhat different, with a network of modules which
either represent single behaviours or combinations of behaviours, and which produce outputs that drive
actuators or influence other modules. In this it is similar toother distributed planning mechanisms such
as those of Maes (1991a) and Rosenblatt & Payton (1989).
BOD could be used as a fusion architecture if several agents are running their own plans in the
robot at the same time, but the POSH plans would not really be mediating the interaction between the
behaviours since it would unless the trigger for one action came from the behaviour being run from
another agent. However the BOD methodology does not encourage this type of design, since there is no
consideration of assigning different tasks to different autonomous agents in the methodology.
2.6.2 Animal behaviour
Several years previously Tyrell (1993) had undertaken an evaluation of several different action selection
mechanisms using a complex simulation of an animal in an environment and the decisions it would need
to make. The simulation provided a landscape for the animal and a set of six goals (such as obtaining
∗My thanks to Martin Proetzsch for clarifying these issues for me
16
sufficient food and avoiding hazards), involving 15 sub-problems. He implemented five action selection
mechanisms (including Rosenblatt & Payton’s artificial neural network (Rosenblatt & Payton, 1989),
Maes’ spreading architecture (Maes, 1991a), and a hierarchical decision structure).
Since most of them were not completely specified, Tyrell extended and modified them to suit the
requirements of his simulation, in particular producing anextended version of the Rosenblatt & Payton
architecture (Extended Rosenblatt & Payton, abbreviated to ERP), which was the most successful of the
five he tried.
Bryson implemented an Edmund action selection mechanism inTyrell’s simulation and compared
it against his extended Rosenblatt & Payton architecture (Bryson, 2001b). The new action selection
mechanism significantly outperformed Tyrell’s ERP in threeof the four variant simulation environments
that Tyrell had used (ones where food was scarce), and under-performed it in one.
Bryson et al. (2006) describes a demonstration of BOD applied to an agent simulation environment.
It used the MASON discrete-event simulation and visualisation environment (Luke et al., 2005) to
simulate sheepdogs herding sheep. The action selection wasimplemented using the jyPOSH engine,
a port of Kwong’s pyPOSH engine (Kwong, 2003) to Jython to permit it to interface with the Java
MASON code. The action selection plan is for each sheep to move towards the other sheep if is too far
away and move away from a dog if the dog is too close. The presence of a dog influences the sheep’s
affinity to other sheep, reducing “too far away” so that the flock compacts. The dogs’ default behaviour
is to do nothing if the flock is not too widely dispersed, otherwise it will select the closest sheep and
move towards it.
The BOD/MASON environment has been used in teaching demonstrations, where students can
dramatically change the behaviour of the antagonists – for example modifying the sheep behaviour so
that they scatter on the approach of a dog to mimic the behaviour of deer.
An early application of BOD was a simulation of the social interactions between primates, with
three behaviours initially implemented (Bryson & Flack, 2002). The first of these was to groom, with
several states such as selecting a grooming partner; the second was to explore new locations and the
third was more complex interactions with other primates. Subsequent work has been done using the
BOD/MASON environment to simulate the behaviour of macaquemonkeys.
It has also been used for modelling the learning of transitive inference in primates (Bryson & Leong,
2007), and the work repeated using ACT-R as a comparison withBOD as an undergraduate project.
BOD has more recently been used in a large scale military logistics simulator to model the actions of
soldiers in constructing facilities during military deployments (Perkins, 2006), although the published
details are sketchy. BOD is occasionally cited in the discussion of reactive architectures, in particular
noting its separation of control and behaviour, for examplein Chen & Chang (2008).
2.6.3 Non-player characters in computer games
Behaviour Oriented Design has been used to control non-player characters in computer games. The
earliest work was to combine it with the Ymir three-layer architecture (Thorisson, 1999) to produce the
Spark of Life autonomous character architecture (Bryson & Thorisson, 2001).
Andy Kwong (2003) created a simple bot for the commercial (Epic Games Inc.) Unreal Tournament
17
first person shooter game. It interfaces with the game using the Gamebots (Kaminka et al., 2002) virtual
reality platform which provides TCP/IP socket-based communication between remote agents and a
computer running the Unreal Tournament server. The human player GUI interacts directly with the
Unreal Tournament server using its normal socket based interface.
Kwong’s original implementation was to provide a proof of concept – his bot was capable of wan-
dering around the virtual world avoiding obstacles and following other players. Partington & Bryson
(2005) describes an extension of this work with the full implementation of a bot for the “Capture the
Flag” mode of Unreal Tournament 2004, as an evaluation of using Behaviour Oriented Design. This
game mode (one of ten) has two teams competing to reach the opponent team’s base to capture its flag.
The outcome of the evaluation was that BOD was highly beneficial although it did not solve all of
the difficulties in Unreal Tournament bot development (suchas navigation, and locating bugs in calling
the Gamebots’ API) and there were some minor problems related to the methodology – notably the
absence of a proper POSH IDE to help with debugging.
Pogamut (Brom et al., 2006) is a comprehensive platform for implementing game bots in the Un-
real Tournament, and includes a POSH engine, ABODE, an editor for POSH plans, and some basic
behaviours.
The current version, Pogamut 3 (Gemrot et al., 2009), interfaces to the Unreal Engine 2004 Runtime
engine to allow virtual worlds other than the Unreal Tournament combat arena to be simulated, and again
includes the jyPOSH action selection engine. The authors ofPogamut 3 are currently re-implementing
the POSH engine in Java.
BOD’s iterative methodology, although apparently not POSH, was used to generate the behaviour
trees for controlling an autonomous player in the commercial strategy game DEFCON (Lim et al.,
2010).
Work has been done in other areas, for example Bryson et al. (2002) and Bryson et al. (2003) exam-
ine how BOD could be used to implement agents for discovering, selecting, composing and using web
services, and for composing new functionality from combining services described using the semantic
web language DAML-S (Paolucci & Sycara, 2003). A comparisonof the different techniques which
have been used in this problem domain, including BOD, may be found in Kumar & Mishra (2008).
18
Chapter 3
Background to the work undertaken
The work undertaken was to carry out a small software development project to evaluate the use of BOD
for developing code to run on a robot. The task undertaken by the robot needed to tailored to provide
information about the effectiveness of BOD.
3.1 Benchmarks
An examination of literature was carried out to identify a suitable benchmark task. del Pobil (2006)
reports that there is a lack of systematic benchmarks in robotics research, which means that problems
are often considered “solved” after they have been demonstrated but not rigorously tested. Because
robotics is so varied a wide range of benchmarks are needed.
The majority of benchmarks are in the form of competitions, normally for mobile robots, such as
locating feeders in a maze (Michel & Rohrer, 2008) or soccer playing (Kitano et al., 1997). There are
nevertheless efforts at defining non-competition benchmarks. This includes benchmarks for grasping,
which have some relevance to the work described here.
However, there is a tendency to either focus upon the physical characteristics of the hand, for exam-
ple (Kragten & Herder, 2010), or on higher level tasks which require deliberative planning for example
(Zielinski et al., 2006). Although EURON (European Robotics Research Network) has a manipulation
and grasping benchmark initiative to define benchmarks thiswork is far from complete (EURON, 2010).
A review of existing benchmarks showed that none were ideal for using to evaluate BOD because
they were either too complicated given the time available ortoo specific to a particular area of robotics
such as grasping which would not provide an overview of the suitability of BOD in several areas.
Consequently it was necessary to define a new benchmark task for this evaluation.
3.2 The iCub robot
The iCub robot was selected for this work as representative of a modern robot, in particular the sophis-
tication of the hardware and software. It also has a simulator, which was essential, since access to a
physical robot would be severely limited.
19
The iCub is a humanoid robot with the same proportions as a 3.5year old child (Tsakarakis et al.,
2007; Metta et al., 2008). It is 104 cm high when standing and contains 53 joints. All joints have
position sensors, and in later versions the limbs have forcesensors, and fingertip pressure sensors. The
eye sockets are fitted with low resolution (640x480 8 bit RGB,30 frames per second) miniature cameras
in pan/tilt mounts, and microphones on either side of the head. It also has an inertial platform containing
gyroscopes and accelerometers, and a magnetic compass.
Figure 3-1: An iCub robot
The robot uses an on-board PC104 controller equipped with anIntel Core 2 Duo processor to carry
out low level control functions and to provide a a gateway between a Gbit Ethernet used to communicate
with the control computers, and the multiple CANbuses used to interface to DSPs interfacing to the
robot’s sensors and actuators. CANbus is 1 Mbit/s serial busoriginally developed for automobile use,
and defined in ISO 11898.
The robot is not intended for autonomous operation, and is not therefore equipped with an onboard
power supply, receiving power via an umbilical cable attached to its back. In addition most of the
computational workload is offloaded from the PC104 controller onto standalone computers.
The robot was developed by the RobotCub Consortium, a collaboration between seventeen research
institutes, in the United States, Japan and Europe. The project, to produce a robot which could be used
in cognitive research, was funded by the European Commission for five years, from 1 September 2004
until 31 January 2010. A summary of the project and its aims isto be found in Metta et al. (2010).
Both the hardware design and software are free/open source,having been released with the GNU
Free Documentation and GNU Public License respectively. However the hardware is expensive and
there are only around 20 robots, of which four are located in the United Kingdom, at Imperial College,
and the universities of Plymouth, Hertfordshire and Aberystwyth (which only received theirs in August
2010).
20
A cognitive software suite has been developed (described inVernon et al., 2007), which is based on
the operation of structures in the human brain. The architecture is focused on the robot learning for itself,
which requires it to be equipped with certain behaviours corresponding to some of the innate abilities
of a small child, such as locating faces, visually tracking moving objects and learning by imitation.
The hardware reproduced the human physique since the learning processes of a small child cannot be
mimicked without the corresponding physical abilities (for example its ability to crawl).
There is a fundamental difference between the aims of the RobotCub project development, which
was geared towards developing a tool understanding human cognition by simulating it; and the work
described here, which is merely using the iCub as a robotics platform to evaluate BOD.
3.3 Robot middleware
The software responsible for planning and cognition ultimately communicates with the robot’s hard-
ware. In the simplest robots, this consists of a few sensors and microcontrollers connected to a CPU
via hardware ports. The CPU can interact either directly with these devices or it can also communicate
with them using a device-independent robotic middleware layer. This middleware provides a common
interface to the different components.
This is important in more complicated robots which have a large variety of different devices, and
where the intelligence is distributed across several processors. This middleware provides the same ad-
vantages in robots as operating systems do in general purpose computers in that they simplify commu-
nication and reduce device and network specific dependencies. The higher level cognitive systems are
isolated from the hardware, allowing hardware architecture to change without requiring major changes
to the cognitive layers.
Many other features are often built into the middleware, forexample simulation, so that the higher
level software communicates either with real devices or with a simulation of those devices. The middle-
ware may include its own scripting programming languages and it can include features such as image
processing libraries. Mohamed et al. (2008) examines several different middleware packages and their
capabilities. The middleware may also change the interfaceto the robot, for example by providing in-
verse kinematics so that the end effector positions (i.e. the hand in the case of the arm) may be specified
in a world coordinate system rather than in terms of joint angles; or it may provide logical interconnec-
tion between force sensors and actuators to provide limbs with springiness, so that they act as if they
incorporate a mass-spring-damper-like system (for example Wimbock et al., 2007).
3.4 YARP
The RobotCub Consortium adopted the YARP open source system(Metta et al., 2006) as their middle-
ware, primarily because of a strategic decision to use a software platform with a future life longer than
that of the iCub robot itself. This was in contrast to the majority of robot projects, which are “one-off”
projects after which the hardware and software are discarded. This reasoning is explored further in
Fitzpatrick et al. (2008). YARP was developed for Kismet, a robot head designed in the mid-1990s at
21
MIT to experiment with expressions in communication with humans (Breazeal & Scassellati, 1999).
Since then it has been used in around eight major robot projects, including iCub and the RobotCub
consortium has carried out a considerable amount of development to extend YARP for iCub.
The main component is the operating system library, which provides a communications system in
which the sensors and actuators on the robot are presented asports, and CPU processes can also create
virtual ports. The model follows the Observer object oriented design pattern (Gamma et al., 1995, pages
292–303), allowing any number of observers to register to receive data sent to or from a particular port.
This allows the code to be distributed across several processors with no dependence upon where
it is located. The library also provides platform independent interfaces to interprocess communication
primitives such as threads and semaphores.
In addition to the main library, there is also a signal processing library for handling vision and sound,
a device handler library which implements drivers for some specific hardware and a maths library. Most
of the code is built on top of other packages, for example the operating system library is built around
ACE open source networking library (Schmidt et al., 1993), and the signal processing library relies upon
the OpenCV vision library (Bradski & Kaebler, 2008).
3.5 Build environment
Since the iCub and YARP are developed by the same people they have much in common. They are
both written in C++ and each maintained in their own Sourceforge project – although the iCub project
contains a large amount of hardware documentation as well assoftware. Both projects use an open
source software configuration tool called CMake (Wojtczyk &Knoll, 2008) for generating the make files
used in the build process. CMake allows automatic checks to be made for the presence of components
needed to build the software, and for setting platform dependent parameters such as the location of
various libraries and header files. This is of significance because any project using either the iCub
software directory or YARP must also use CMake.
CMake uses configuration files in the directory hierarchy indicating which components need to be
built and with what compiler or linker options. A CMake configuration file will usually use output
generated by the configuration file in the containing directory to obtain most of its information. Thus
if CMake is configured and run from the top of the directory hierarchy it automatically processes the
CMake files in the sub-directories and configures the caches.Thus once the configuration has been
done it is only necessary to run the Makefile in the uppermost directory for the compilation process to
cascade down to the leaves of the directories building each set of sources in each directory, avoiding
attempting to build any which do not have their dependenciescompletely satisfied.
YARP can be built, and used, independently of the iCub software since it is used in several other
projects; but the converse is not true since the iCub software requires YARP.
22
Figure 3-2: Simulator iCubhand
3.6 Simulator
In common with most robots, the iCub has a simulator (Tikhanoff et al., 2008) which is used in many
of projects associated with it, including ones such as wholebody reaching which cannot be done with a
real robot.
The simulator is implemented below the interface to YARP in the software hierarchy, so that the
planning and cognitive levels are “unaware” that they are running on a simulator.
The simulator contains a rigid-body physics model, ODE (Smith, 2006) and incorporates graphics
and collision detection. It can also simulate rigid, passive objects such as as cubes, balls and cylinders
from which larger objects such as tables and space frames canbe built up. The characteristics of the
objects are very restricted, with only the dimensions and colours being under user control, the mass,
the distribution of mass and the surface texture for exampleare not under user control – since they are
hardwired constants in the simulator. The objects must be rigid since the underlying physics engine
does not support flexible objects. Consequently activitiessuch as folding towels (Maitin-Shepard et al.,
2010) could not be emulated using the simulator.
The simulator differs from the real robot in several respects. The graphics are naively simple, which
is significant for image processing, since the simulated robot cameras “see” an image from the visual
representation of the robots environment provided by the simulator.
With the exception of the face, the components of the robot are modelled as simple cubes, cylinders
and balls. This makes image processing considerably easier, in particular in locating the hand in space
since the hand is very simplistically modelled.
Like most robotics simulators, the iCub simulator does not simulate the mechanics of the robot and
its environment with any great fidelity. For example it does not produce the positional errors that the
iCub has which are caused by slight misalignment of the jointand eye position sensors, slight differences
in axis positions between different robots, and whiplash inthe actuators.
The model of the robot is such that parts of it may intersect with each other. For example it is
possible to pass the fingers of one hand through the fingers of the other.
Nevertheless despite its deficiencies the simulator does allow underlying control logic to be tested,
at least in a closed loop environment. That the images generated are not really representative of the real
robot’s view of the world means that the image processing needs to be modified in moving to the real
robot; not that logic that it influences is faulty.
Similarly moving a limb may result in a slightly different position in the real robot because of
23
Figure 3-3: iCub hands. Thethumb is barely visible onthe right hand because it bentdown and is pointing roughlyinto the picture. The robot ispowered off, so the LEDs onthe back of the right hand arenot illuminated. The brownpatch on the left hand is thepalm touch sensor.
Figure 3-4: iCub right hand.This hand is not equipped withfinger tip touch sensors. Notethe background clutter com-pared to the previous picture,which would complicate imageprocessing, and the illuminatedred and green LEDs.
mechanical imperfections, but in a closed loop environmentthis might mean that the damping needs to
be modified rather than that it will fail.
24
Chapter 4
Work undertaken
4.1 Project plan
The project lasted for about 15 weeks, during which the principal task was to apply BOD in a small
software development task to implement some functionalityin a robot so that an evaluation view could
be made of its suitability for this type of work.
Once an initial task was completed, an assessment of how useful it was and any problems could be
made. The task could then be extended in two ways:
• Extend the robot’s task. This would have two benefits – the first is to increase the experience
with BOD and the second is to see if there are any problems in extending software using it. All
production software is subject to continual modification and extension.
• Get other software engineering experts, but with little experience in AI or robotics, to attempt a
further extension development task using it and obtain their views and opinions.
It was important that the first task was not implemented with an aim of implementing a second
task afterwards, since the purpose of the second implementation was to see if BOD produced software
which could be modified. Most software undergoes significantmodification during its life, and usually
in directions which are not anticipated by the original designers. Frederick Brooks noted as long ago
as 1975 that the requirements of a system will inevitably change over the course of its construction,
verification and use (Brooks, 1995, page 117). It is therefore essential that any software development
methodology should produce a design which is not so inflexible that it cannot be extended in unforeseen
directions.
Several different activities were considered, notably crawling and reaching and grasping. Crawling
was rejected as it would not be practical on a real robot sinceit stresses the components too much.
However reaching and grasping was an ideal task because it was straightforward, did not need any
special equipment and could be done both in the simulator andthe real robot.
Work has already been done using an iCub simulator with neural networks for both vision and
grasping (Macura et al., 2009), and this type of activity forms part of most demonstrations with the real
iCub robots, so was known to be feasible.
25
The chosen task is to visually locate and pick up an object from a table. The object was originally a
blue ball but was changed to a blue cube, as explained later.
The benefit of this task is its extensibility – an original implementation can be extended, and if it
proved easy to implement then further features can be added;or if it is hard then the functionality can
be scaled back.
The task could be extended in several ways, for example:
1. Placing the object back in the same position again.
2. Passing it from one hand to the other.
3. Catching the object
4. Tossing the object in the air and catching it in either the original hand or the other hand.
There is sufficient variety of actions that the task could be used as the basis for other evaluators
to extend since it will be easier for them to extend a task thanwrite one from scratch. The suggested
extensions can all be made cyclic. It is practical to carry out in the simulator because small random
effects can easily be added, for example randomising the initial position of the object, to make repeating
the task worthwhile.
The task was considered to be fairly easy to do, with the greatest difficulty originally envisaged to be
the vision processing for locating the cube and the hand. If this could not be surmounted then it would
be hard to obtain results of the BOD evaluation.
It was intended that most of the work would be done using the simulator, with the code ported to
a real robot if time permitted. The objection to using a real robot from the outset was its availability
– there are only four in the UK and the closest ones are two hours journey away; and there would be
pressure from those universities’ own staff to use the robot. Therefore it was considered that access to
the real robot would be severely limited.
There was also the opportunity to use a real iCub at a summer school, but a robot would be accessible
there would still be competition from the other 50 or so attendees at the summer school for access, and
it to be held at a fixed date around the middle of the project. Nevertheless it may have presented an
opportunity to port the code if it was ready in time.
Porting to a real robot was not critical for the work since in assessing BOD the differences between
the simulator and the robot are not really consequential. Problems encountered using either only the
robot or only the simulator are likely to be due to particularissues with the platform rather than prob-
lems with BOD since the differences between the robot and thesimulator are small compared to the
differences between two types of robot. Nevertheless porting the code to a real robot would produce
a useful demonstration which would dramatically show that BOD was a useful tool in robotics, should
this be the outcome of the evaluation.
4.2 Initial evaluation task
The initial task was to program the robot to lift an object from the table. This was a lengthy task because
it included building the infrastructure needed to use the robot from a POSH plan. During development
26
the task was slightly extended so that the robot would pick upthe cube and drop it at arm’s length on the
right hand side of its body. The reason for the extension was to allow the program to be run cyclically
– having disposed of a cube the robot could request another one and repeat the process indefinitely.
This made soak testing easier since the simulator and POSH plan could be left running and periodically
examined to check for problems.
(a) Simulated robot and table (b) Robot leans forward to extendreach over table and raises arm side-ways, to avoid striking the table
(c) Robot determines height of table bygradually lowering hand onto it
(d) Upon touching table robot raiseshand
(e) Robot moves hand and body clear oftable and requests a cube to be droppedonto the table
(f) Robot locates position of cube visu-ally and moves hand over table
(g) Robot moves hand to cube (h) Robot grasps cube (i) Robot picks up cube
(j) Robot discards cube (k) Moves out of the way of table, andrequests another cube to dropped ontoit
Figure 4-1: The original task – picking up a cube from a table
27
4.3 Architecture
The software is split in four logicals components.
4.3.1 POSH engine
At the highest level the POSH engine parses and processes thePOSH plan to carry out the highest
priority goal. The engine used is the jyPOSH Python one originally written by Andy Kwong. This
engine was chosen because it was easy to interface to C++ and was considered to be a canonical POSH
engine.
4.3.2 POSH plan
The POSH plan is read by the POSH engine and controls the behaviour primitives. The format of the
plan is mandated by the jyPOSH engine. Multiple plans can be used to control multiple agents, but for
this project there was only one plan. Further discussion of the plan is in Section 4.4.5.
4.3.3 Behaviours
The behaviours are written in Python since development in a weakly typed scripting language with
automatic memory management is much quicker for rapid prototyping than using a compiled strongly
typed language such as C++. Bryson herself also noted that she found it five times faster to program
behaviours using a scripting language, Perl 5, rather than C++ (Bryson, 2001a, page 70).
However the iCub software and YARP are implemented in C++. A minimal YARP interface to
Python exists, and was used for some of the early work in demonstrating connectivity between Python
behaviour code and YARP, but it lacks the functionality required for convenient use. In particular it
provided only the network communication features of YARP, and not the more sophisticated interfaces
to device controllers.
Consequently a better interface to use the iCub robot was written which could be invoked from
Python, and was implemented as a set of classes for controlling individual subsystems in the robot, such
as each arm or the torso. It provided a unified interface to thedirect control of joints, or positioning
the hand in Cartesian space using inverse kinematics, whichare implemented quite separately in the
underlying iCub software.
It was also necessary to interface with code which carried out processing of images to locate the
object and the robot hand, and to control the simulator world, for example to generate the table, and to
request a new object be placed on the table once the previous one had been removed. All of this code
was written in C++.
4.3.4 Interface between Python and control classes
The specification and design of the classes followed from thespecification of the behaviours. It was
clear, for example, that in moving the hand to a particular location then individual joints would need
to be moved. Even though no consideration of the extension task was being made, the symmetry of
28
the robot implied that the same functions should work for theleft arm as for the right arm with just a
change in parameter. Once a function to move a particular joint in a particular arm was considered then
the same function should be able to move, for example, a jointin the leg by changing the parameters.
Since the behaviours were being implemented incrementallyit was not obvious how the partitioning
of the control of the robot into classes should be presented to the behaviours, since this would not
be known until several behaviours had been implemented. Consequently the interface to the robot
was presented as a collection of functions. However it was worthwhile performing some partitioning
between, for example commands to the robot and commands to carry out image processing, since there
was a clear and obvious separation in functionality. Consequently the interface presented to the Python
behaviours was represented as three classes, one to controlthe robot, one to handle image processing
and one to handle the environment around the robot.
Each one of these classes was implemented as a C++ module withsome code allowing it to be
called from Python. The classes themselves had no attributes, but each one calls public methods in a
corresponding Singleton class (Gamma et al., 1995, pages 127–134) so that all behaviours communicate
with a single instance of each underlying class. This slightly complex arrangement was needed for
implementation reasons. The classes used from Python needed to be as simple as possible to make the
interface to Python easy to generate automatically. The classes are:
1. ICubInterface – provides the interface to the robot. As development of behaviours proceeded
more functions were added to this class as required.
2. VisionInterface – This provides image processing, locating the cube and the hand.
3. WorldInterface – This provides the interface to the simulated environment, in particular the pro-
vision of cubes for the robot to pick up.
4.3.4.1 Generating the interface code
The interface layer between the Python code and the C++ code was generated using an automated
code generation tool. The Simplified Wrapper and Interface Generator (SWIG)), is a code generator for
automatically interfacing languages (Swig, 2009) together – it works in the C++ case by processing C++
header files and using these to generate a Python module whichin turn calls a SWIG-generated C++
module to perform out the data transformations needed to pass parameters between the two languages.
It was considered simpler, faster and less error prone to useSWIG than to generate the interface layer
manually, particularly in making changes once an initial layer had been produced.
However the SWIG code generator cannot handle all C++ declarations, for example templates can
be problematic, which caused difficulties with some YARP header files.
The interface to the robot is written in C++ because that is the native language used by YARP and
the iCub software such as the iKin library, and OpenCV. This interface provides a convenient set of
Python-callable functions and is needed since Python does not have native support for the types of
C++ structures which YARP and the iCub software uses. It would have been possible to make a direct
interface between YARP and the iCub software but it was felt better to provide an interface which would
be easy to use from Python.
29
4.3.5 iCub control classes
The control classes underneath the Python behaviours are all implemented in C++, and reside in the
iCubInterface namespace to avoid possible clashes with other objects of the same name. three classes
make up the external interface, as outlined above.
4.3.5.1 ICubInterface
The iCub interface class provides a Python-callable interface to allow the iCub to be controlled. The
iCub robot is divided into different sub-systems both physically and in its own software, and this is
reflected in the design of this interface. The arm, the head, the torso and the legs form subsystems.
Most of the functionality for controlling the limbs is implemented in a superclass, SubSystem, and this
maintains a list of the different limbs. When a command is received to move a limb, it searches through
the list and asks each entry whether it has the capacity to process the command. All those which have
the capacity are sent the command.
This allows, for example, both legs to be moved to the same joint angles simultaneously, by speci-
fying BOTH ARMS instead of, for example, LEFTARM.
Commands to move joints can be specified to operate asynchronously or synchronously, in which
case the robot blocks until all movement of that limb has completed.
The legs are normally disabled, but can be enabled by an appropriate method call. They have been
implemented for completeness and to verify that the model iscapable of handing inverse kinematics for
disparate limbs.
The kinematics is implemented for the arm and the leg. These incorporate the iKin kinematic chains
for the appropriate limb, but use a separate class, Kinematics, to carry out inverse kinematic calculations
using iKin as it provides uniform error handling for solver problems. Kinematics could be implemented
for the head and torso but there is little point at present since explicit joint movements are adequate for
the cube handling tasks. Since iKin uses radians whilst the interface to the iCub generally uses degrees,
methods are provided to specify joint angles in either degrees and radians. Because of the possible
confusion two methods are provided to set the angle, and two to read the angle, and explicitly indicate
in their names whether they use radians or degrees, for example setJointDegrees() .
The touch and video systems implement interfaces to the touch sensors and cameras.
The Component superclass was used by all major classes, and provided the objects with a name to
be used in error messages. The JointDetails class held the name and limits of a joint, and along with a
method for checking that the limits were not being exceeded by a command.
4.3.5.2 Program flow through Python and C++
The basic pattern of execution through this module is as follows:
• A command to move a hand is issued by the Python behaviour:
s e l f . iCub . s e t L o c a t i o n ( i c . RIGHTARM,
−0.2 , 0 . 0 1 ,
s e l f . ag en t . F i n d i n g Tab le . t a b l e Z +
30
Figure 4-2: ICub Interface
s e l f . wor ld . ge tCubeS ize ( ) +
3 ∗ s e l f . HOVERHEIGHT ABOVE CUBE,
−0.4 , 0 . 7 , 0 . 0 6 ,−math . p i∗0 . 9 ) # R o t a t i o n
• It is passed by the SWIG generated layer to the ICubInterface static class. This converts the
parameters from the format used by Python to the format used internally within the C++ code, for
example a Location vector instead of a list of values. The equivalent call is then made to the ICub
singleton, which signals failure by generating exception,which is converted to a character string
and rethrown for the Python interpreter. The main reason forthe separate ICub class is to insulate
SWIG from the complexities of the YARP header files which the ICUB header file includes.
vo id I C u b I n t e r f a c e : : s e t L o c a t i o n ( SubSystemId subSystemId ,double x ,
double y , double z , double xRot ,
double yRot , double zRot ,
double r a d i a n s , bool i g n o r e O r i e n t a t i o n ,
bool synchronous ){
31
t r y {
Lo ca t i o n l o c a t i o n ( x , y , z , xRot , yRot , zRot , r a d i a n s ) ;
ICub : : i n s t a n c e ( ) . s e t L o c a t i o n ( subSystemId , l o c a t i o n ,
i g n o r e O r i e n t a t i o n , synchronous ) ;
}
ca tch ( e x c e p t i o n e ) {
throw e . what ( ) ;
}
}
• The ICub class has a Standard Template Library set (Josuttis, 1999, pages 175–193) of SubSystem
objects, so that it does not have to know the details of particular limbs in the robot. It asks each
subsystem if it is capable of fielding the request, and if so then it commands it to do so. For
commands such as setting the joint angle, it always asks all subsystems in the set, so that both
arms can be moved at the same time (e.g. if the subsystem ID is set to BOTHARMS). It does
not do this for setting the location since this would result in a collision.
vo id ICub : : s e t L o c a t i o n ( SubSystemId subSystemId ,
cons t Lo ca t i o n& l o c a t i o n ,
bool i g n o r e O r i e n t a t i o n ,
bool synchronous )
{
f o r ( se t<SubSystem∗>:: i t e r a t o r i t = subSystems . beg in ( ) ;
i t != subSystems . end ( ) ; i t ++){
i f ( (∗ i t )−>match ( subSystemId , JOINTANY , f a l s e ) ) {
(∗ i t )−>s e t L o c a t i o n ( l o c a t i o n , i g n o r e O r i e n t a t i o n , synchronous );
re turn ;
}
}
throw ICubExcep t ion (t h i s ,
” SubSystem no t c a p a b l e o f s e t L o c a t i o n ( ) ” ) ;
}
• In the case of an arm movement, the setLocation() method in the Arm class directs the processing
of the inverse kinematics calculation and the joint movements.
vo id Arm : : s e t L o c a t i o n (cons t Lo ca t i o n& l o c a t i o n ,
bool i g n o r e O r i e n t a t i o n ,
bool synchronous ){
i f ( ICub : : ve rbose ( ) ) {
CLOG << getName ( )<< ” t o ”
<< l o c a t i o n . t o S t r i n g ( i g n o r e O r i e n t a t i o n )<< end l ;
}
iK inCha in∗ ch a in = k inema t i csCha in−>asCha in ( ) ;
/ / We don ’ t use t h e t o r s o i n p o s i t i o n i n g t h e hand , bu t we have
/ / t o t a k e i t s p o s i t i o n i n t o accoun t
cha in−>r e l e a s e L i n k ( KINTORSOPITCH ) ;
cha in−>b lockL ink ( KIN TORSOPITCH , ICub : : i n s t a n c e ( ) . g e t J o i n t R a d i a n s (TORSO, TORSOPITCH ) ) ;
32
cha in−>r e l e a s e L i n k (KINTORSOROLL ) ;
cha in−>b lockL ink (KIN TORSOROLL , ICub : : i n s t a n c e ( ) . g e t J o i n t R a d i a n s (TORSO, TORSOROLL ) ) ;
cha in−>r e l e a s e L i n k (KINTORSOYAW ) ;
cha in−>b lockL ink (KIN TORSOYAW, ICub : : i n s t a n c e ( ) . g e t J o i n t R a d i a n s (TORSO, TORSOYAW ) ) ;
. . .
i f ( synchronous ){
wai tForMot ion ( ) ;
}
}
4.3.5.3 VisionInterface
The Vision Interface is responsible for locating the hand and the cube. It obtains still images from the
iCub cameras and uses OpenCV to help it extract the information that it needs.
Figure 4-3: Vision Interface
4.3.5.4 WorldInterface
The World Interface provides a Python-callable interface to the simulator, allowing manipulation of the
robot’s simulated environment. With a real robot, it would be replaced by tasks carried out by a human,
so would be changed to display requests on the computer monitor.
Figure 4-4: World Interface
The tasks which the simulator needs to perform are to create the table, request a cube to be dropped
onto the table in either a random or fixed position and to request the cube be removed. Two slightly
unusual methods are to maintain a counter of the number of cubes which have been dropped onto the
33
table so that the task can stop after a finite period and request the magnet grab feature of the simulator
be enabled. The reason that the latter is here and not in the ICub interface is that it connects the hand
to a specified cube, so has reason to be implemented in either.It was put into the World interface partly
because the ICub interface has no conceptual knowledge of objects in the environment, and would have
to communicate with the World interface to obtain the current cube number, and partly because it is a
feature of the simulator rather than the real iCub.
Since the interface does little more than construct and sendcommands to the simulator on receipt
of method calls from the Python code, the structure is very simple. The Python code uses the WorldIn-
terface class, which in turn constructs the World singletonand makes corresponding method calls on it.
The World singleton constructs the messages to the simulator and sends them, and checks the responses.
4.3.6 Coordinate systems
A quirk of the simulator is that the coordinate system which it uses for specifying object positions is
not aligned with the coordinate system for the kinematics ofthe real robot. However it is parallel to
it (i.e. the axes are in the same direction) and uses the same units. Of course in principle the robot
can move, which would change the orientation of its kinematics frame, but this does not happen in the
evaluation task. The robot’s vision system is not aligned either, partly because the head can move with
respect to the origin of the robot’s axis, and partly becausethe x axis is from left to right in a camera
image, and from front to back in the robot kinematic frame. The control classes do not try to map these
three coordinate systems into a single uniform one, as it is much simpler to operate in using whichever
coordinate system is appropriate for each particular activity.
4.3.7 Coping with errors
Within the C++ code error handling was done by raising an exception, for which a dedicated class was
created,ICubException , and which was passed up to Python. An error message, highlighted in red
by sending the appropriate escape sequences to the terminal, is displayed when the exception object was
created.
Python cannot natively handle C++ exception classes in its exceptions, and consequently the in-
terface layeriCubInterface , converts them to C strings before raising them as exceptions to the
Python interpreter.
Exceptions are generally rare, and so only needed to be explicitly handled in the Python code in
certain places where they are more likely to occur, such as the hand moving out of sight because the
cube has been dropped into a position which cannot be reached.
The only major area where exceptions are not used is in timeouts waiting for joints to move to
a specified angle. Instead a ten second timeout is used, afterwhich the program continues assuming
that the joint has moved to the necessary position. This works because the hand is constantly moved
under visual feedback, and its failure to arrive at a specificlocation will simply mean that the movement
command sent to the hand will be slightly different than if ithad arrived at the location.
34
4.4 Using the BOD Methodology
The previous section describes the code immediately below the layer of behaviours, and which the
behaviours use to interact with the iCub simulator. As such they do not have relevance to BOD, except
that the requirements for their design flowed out of the BOD methodology used to create the behaviours.
The actual steps in the development of the behaviours are part of the results rather simply consid-
ering the final product. When I started I had little experienceof BOD confined to looking at some of
the existing code and writing a simple POSH plan, and no experience whatsoever of robotics although
I do have considerable experience as a software engineer in other subject areas. In this sense I can be
considered a naive user.
Behaviour Oriented Design, as described earlier, is a software development methodology for im-
plementing the high level control of autonomous agents. These agents have a set of prioritised internal
goals which they are constantly striving to achieve. Each goal is made up of several subgoals which
must be attained to achieve this higher level goal. Once thisgoal is satisfied the agent attempts to sat-
isfy the next lower priority high level goal. Each goal has a trigger associated with it, that determines
whether the goal has been achieved. If the goal has been achieved then it is ignored and the next higher
priority goal examined.
The development steps for the first implementation using Behaviour Oriented Design were as fol-
lows.
4.4.1 High level specification
The first stage of using the BOD methodology is to specify in general terms what the agent is to do. In
the original scheme, the robot merely had to locate the ball on the table and pick it up.
4.4.2 Generate likely sequence of actions
The next step was to formalise this into a sequence of likely actions.
1. The robot needs to find the object
(a) The robot moves its hands clear of the table
(b) The robot returns to a default pose
(c) The robot sees the object
2. The robot needs to pick up the object
(a) The robot ensure its hand is open
(b) The robot moves its hand over the table
(c) The robot moves its hand to the object
(d) The robot closes its hand around the object
(e) The robot lifts its hand, holding the object, and rotatesit
35
4.4.3 Generate a list of sense and action primitives
The original set of senses and actions were as follows.
1. The robot needs to find the object
(a) The robot needs to move its hands clear of the table
(b) The robot needs to return to a default pose
(c) The robot needs to see the object
2. The robot needs to pick up the object
(a) The robot needs to ensure its hand is open
(b) The robot needs to move its hand over the table
(c) The robot needs to move its hand to the object
(d) The robot closes its hand around the object
(e) The robot lifts its hand and rotates it so it is holding theobject
4.4.4 Identify the state necessary to enable the described primitives and drives
The robot needs to know:
1. Where the hand is over the table
2. Whether the robot is in a known pose where it can see the object
3. Where the hand is
4. Where the object is
5. Whether the hand is adjacent to the object, which is relatedto the previous elements
This developed into a behaviour library. There was only one action plan needed, since coordinating
multiple simultaneous actions for the robot would be hard, and the two basic elements – vision and
kinematics – were intertwined. The robot could not see the object if the hand was in the way (or would
estimate its position based on the parts of it that it could see), and the hand could not do anything useful
without input from the vision. For these reasons it was decided to use a single plan and behaviour
library.
Moving the hand to the table was seen as a different type of operation to moving the hand around
the table as it was intended that inverse kinematics would not be needed. Hence moving the hand to
the table was seen as a sequence of explicit joint movements to bring the hand from beside the robot
to a position over the table. Removing the hand again was seenas a similar set of movements. Once
the hand was over the table small iterative movements to joints – probably with some learning during
execution – would pilot it around the table.
1. The robot needs to move its hands clear of the table
36
2. The robot needs to return to a default pose
3. The robot needs to find the object
4. The robot needs to pick up the object
5. The robot needs to ensure its hand is open
6. The robot needs to detect if its hand is over the table
7. The robot needs to move its hand over the table
8. the robot needs to sense if its hand is over the object
9. The robot needs to move its hand to the (at that stage) object
10. The robot closes its hand around the object
11. The robot needs to sense if it holding the object
12. The robot lifts its hand and rotates it so it is holding theobject
4.4.5 Prioritise the list of goals
The initial highest priority goal was to locate the object. Once the object had been located the next stage
was to pick it up. Within each of these two goals the highest priority was really to prevent the robot from
hitting the table. Consequently opening the hand was important so that the fingers were inline with the
palm so that the arm could be safely moved above the table without the fingers sticking out sideways to
hit it.
From this the initial POSH plan was created, as follows. Looking back at it one feature is that the
elements tend to be ordered in the sequence in which they needto be done rather than the priority.
Generally they need to be ordered in the reverse order so thatactions normally performed later in a
sequence are triggered if there is an opportunity to carry them out without the preceding steps.
(
# Find t h e b a l l . Move t h e hands ou t o f t h e scene and then
# look f o r t h e b a l l on t h e t a b l e
(C f i n d−b a l l (MINUTES 1) (GOAL ( b a l l f o u n d ) )
(ELEMENTS
(
( move−hands−away (TRIGGER ( ( h a n d sc l e a r 0 ) ) ) m o v eh a n d s c l e a r )
( r e l a x (TRIGGER ( ( r e l a x e d 0 ) ) ) r e l a x )
( search (TRIGGER ( a lways ) ) s e a r c hf o r b a l l )
)
)
Note that there is a bug in the plan, which is that the hand openis the highest priority in the pick-up-object competence, butlater in the competence the hand is closed. Once this has been achieved the hand would have been opened again and so the robotwould have oscillated between opening and closing the hand.Nevertheless this was the first set of behaviours which started to beimplemented
37
)
# Move hands on to t h e t a b l e .
(C move−hand−to−table (GOAL ( ( h a n d o v e r t a b l e ) ) )
(ELEMENTS
(
( move−hands−away (TRIGGER ( ( h a n d sc l e a r 0 ) ) ) m o v eh a n d s c l e a r )
( r e l a x (TRIGGER ( ( r e l a x e d 0 ) ) ) r e l a x )
( move−hand−to−table (TRIGGER ( ( h a n do v e r t a b l e 0 ) ) ) m o v e h a n d t o t a b l e )
)
)
)
# Pick t h e b a l l up . Move t h e hand t o t h e t a b l e , t hen t o where t h e
# b a l l i s and p i c k i t up
(C pick−up−bal l (MINUTES 1) (GOAL ( ( h o l d i n g b a l l ) ) )
(ELEMENTS
(
( open−hand (TRIGGER ( ( handopen 0 ) ) ) openhand )
( move−hand−to−table (TRIGGER ( ( h a n do v e r t a b l e 0 ) ) ) m o v e h a n d t o t a b l e )
( move−hand−to−ball (TRIGGER ( ( h a n da d j a c e n t 0 ) ) ) m o v eh a n d t o b a l l )
( c lose−hand (TRIGGER ( ( h a n da d j a c e n t 1 ) ) ) c l o s eh a n d )
( ho ld−ba l l (TRIGGER ( ( a lways ) ) ) r o t a t eh a n d )
)
)
)
# Dr i ves
# Top p r i o r i t y − f i n d t h e b a l l
# Then p i c k i t up as second p r i o r i t y
(SRDC iCub (GOAL ( ( h o l d i n g b a l l ) ) )
(DRIVES
( ( found−ba l l (TRIGGER ( ( b a l l f o u n d 0 ) ) ) f i n d−b a l l ) )
( ( g e t−b a l l (TRIGGER ( ( h o l d i n g b a l l 0 ) ) ) p ick−up−bal l ) )
)
)
)
4.4.6 Iterative Development
Development of the action and sense primitives started after the definition of the action plan. The first
ones implemented were those triggered at high priority, with the lower priority ones left as dummy code;
since this task was a sequence of actions which tended to needthe preceding high priority goal to be
achieved before the next one was worthwhile. For this reasona strict POSH implementation seemed
most appropriate.
Although names changed during the development, the eventual plan was not much larger, and similar
to the original one. Three major enhancements occurred at this level, each of which was reflected in the
addition of a drive.
38
• The table height was assumed not to be known precisely, but fall within certain limits. This was
to allow the code to be moved to a real robot, with a real table which will not have an easily varied
height. A drive called “calibrate” was added to find the height of the table by lowering the hand
until resistance was felt.
• Improved error handling was added. There is a fail flag which causes the system to reset.
• A timeout was added. The robot has two minutes to pick up the cube and discard it. If it has not
achieved this within two minutes it requests the cube be replaced with another. This handles error
conditions such as the cube falling into a position where thehand cannot be located over it using
both cameras.
• The behaviour was extended to remove the object from the table, so that the task could be made
cyclic for easier testing and debugging. If the robot does not detect an object on the table in
“find-cube” then the robot requests one to be provided.
(have not found table)⇒ find the table
(error occurred)⇒ reset robot and ask for new cube
(timeout occurred)⇒ reset robot and ask for new cube
(holding cube)⇒ remove cube
(not found the cube)⇒ find cube
(not holding the cube)⇒ pick up the cube
or in .lap file format:
; D r i ves
; Top p r i o r i t y − make su re t h e ro b o t knows t h e h e i g h t o f t h e t a b l e
; Next p r i o r i t y − f i n d t h e cube
; P ick i t up as second p r i o r i t y and remove i t
(SRDC iCub (GOAL ( ( s cube number 1 0 ) ) )
(DRIVES
( ( c a l i b r a t e (TRIGGER ( ( st a b l e f o u n d 0 ) ) ) c a l i b r a t e ) )
( ( f a i l e d (TRIGGER ( ( s f a i l e d 1 ) ) ) r e s e t− t a b l e ) )
( ( t imed−out (TRIGGER ( ( s t i m e o u t 1 ) ) ) r e s e t− t a b l e ) )
( ( remove−cube (TRIGGER ( ( sh o l d i n g c u b e 1 ) ) ) remove−cube ) )
( ( found−cube (TRIGGER ( ( sc u b e f o u n d 0 ) ) ) f ind−cube ) )
( ( get−cube (TRIGGER ( ( sh o l d i n g c u b e 0 ) ) ) pick−up−cube ) )
)
)
Grasping went through several iterations, including beingimplemented in C++ and later Python.
This was done because it became clear that it was a single atomic operation involving a sequence of
steps, and because of concern not to damage the robot by applying pressure on the fingers for too long,
and consequently the wish to put the sequencing of the operations into a localised loop.
Once it was working, and the mechanics of the fingers better understood, it was clear that it did not
need to be a tightly constrained sequence, and it was consequently split out into a POSH competence
again.
39
1. The robot needs to move its hand clear of any area that the table might be
(a) action – move hand clear
(b) action – find the table
(c) sense – touch table
2. The robot needs to look for an object on the table
(a) action – move hand clear of table
(b) sense – find object position
(c) action – request object
3. The robot must move its hand to the object
(a) sense – find hand position
(b) action – move hand towards object
4. The robot must grasp the object
(a) action – check hand height
(b) action – set fingers closing
(c) sense – touch
(d) action – monitor fingers and start/stop them
5. The robot must take its hand away, holding the object and drop the object.
(a) action – raise object slightly
(b) action – swing arm away from table
(c) action – drop object
6. The robot must pick the object up.
(a) grasp object
7. The robot must take its hand away and drop the object when itis clear.
(a) remove object
At the same time a coherent underlying library used by the behaviour was developed.
40
4.5 Vision processing
Since the task was to locate the object visually and move the hand to it to pick it up, image processing
was a key part of it.
The robot is equipped with a pair of cameras producing 640x480 pixel, 8 bit RGB images and these
are reproduced in the simulator, where the interface to themgenerates low resolution images derived
from the viewpoints presented by the directions in which thesimulated cameras are pointing.
An expected difficulty was accommodating with the complexities of computer vision, for example
locating the object and coordinating the movement of the hand in relation to it. However provided the
simplest approaches, for example using a distinctly coloured object so that it could be easily picked
out from the background were used, rather than try to build upan internal symbolic representation
of the scene within code. This type of ad-hoc application-specific specialisation is generally called
“lightweight vision” and has been markedly successful in numerous projects such as the mobile robot
described by Horswill (1995). However the image processingchoices do not affect the project outcome
since they are independent of BOD.
4.5.1 Lightweight vision
In this approach only sufficient processing is done to carry out the task in the specific environment
without any generalisation to handle any other environment. This allows the developer to greatly sim-
plify the development effort required for image processing. Horswill used a resolution of monochrome
camera from which he extracted images of only 64 x 48 pixels tosuccessfully allow a robot to navigate
around an office building. The very low resolution, and limited image processing, worked because the
environment in which the robot operated was tightly constrained to an office area with well understood
and limited features.
It is thought that even animals also use this type of limited technique frequently, largely to reduce
the cognitive overhead in undertaking a variety of tasks. This is done, for example, to reduce the mental
effort, and the resulting fatigue, or to allow short-term memory or attention to be applied to a different
aspect of the task, or to permit the task to be done more quickly.
An example quoted by Clark (1999) is that it was once thought that a fielder in a ball game carried
out trigonometric calculations to plan their approach and position to catch a ball. It is now understood
that the fielder learns to adjust their running until the ballappears to be travelling in a straight line rather
than an arc. Clark and many others (notably Brooks, 1991) argue that much of animal cognition in more
complex animals, and all in simpler organisms, is based uponthe evolution of a tight coupling between
sensory and motor capabilities of an animal and its interactions with its environment rather than a higher
level abstract intelligence.
4.5.2 Visually guided grasping
Image processing was initially identified as one of the hardest areas. However in the event it turned out
to be relatively easy, albeit requiring some experimentation. A very simple approach worked acceptably
well, and a slightly more sophisticated approach worked very reliably.
41
The simplicity was largely due to the great deal of work put into image processing over the past
thirty years, which has resulted in effective algorithms being developed and made readily available in
software, along with the development of the OpenCV package.
There are two aspects to the vision processing:
• Locating the object
• Locating the hand
The hand in the simulator is very simple, and so can easily be recognised. In the real robot hand
recognition is a much harder problem because it changes its appearance with pose and lighting condi-
tions, and even different robots have different hands depending upon whether, for example,the touch
sensors are installed.
It is necessary for the robot to see the hand so that it can guide it towards the object to be picked up.
In many cases the hand moves in 3 dimensions, and not parallelto a plane in two. This causes
several difficulties in practice, partly because of errors in locating the hand because of actuator errors.
It is also difficult to locate the hand visually, partly because of its complexity and partly because of the
errors in the eye positioning.
The solution often adopted in the real robot is to stick a visual marker to the hand and use the
ARtoolkit (Billinghurst et al., 2010) package to determinethe hand position. This searches the image
for the pattern on the marker and uses its size and orientation to determine object position. However
this also requires careful camera calibration using the marker.
Even without this, the back of some versions of the iCub hand is now fitted with a red and green
LED to show orientation.
This package is widely for other applications outside of theiCub although it does require careful
calibration of the camera. It is noteworthy that the vergence of the cameras is rarely used to determine
distance, because of the inaccuracies in the camera positioning and poor resolution of the position
sensors.
Having seen the hand there are two approaches possible, to either map the object from the visual
frame into the kinematics one and then command the hand to move to its position in the kinematics
frame, or to operate entirely in the visual frame.
4.5.2.1 Conversion of visual frame to kinematics frame
In this scheme, the hand position is mapped from the visual coordinate system into the robot’s kinematic
coordinate system. This would allow the hand position to be directly specified using the robot kinemat-
ics, and was the initial approach taken. Since the hand is moving over a flat table, with a constant Z
in the robot’s frame of reference, the mapping only needs to be in two dimensions – the plane of the
table surface. The iCub software provides a package to help do this, the eyeToWorld package. However
the problem was sufficiently simple that it was decided not touse this package, and instead locate the
hand in two places in the visual frame, and from these work outthe mapping of the vision X and Y
coordinates to the kinematics frame. In the event the approach was dropped following discussions with
Giorgio Metta, an experienced iCub developer, who pointed out that this would not work as well as
42
direct visual guidance on a real robot, since small errors inthe joint positions would be magnified to
significant errors in the hand position.
Similarly, the vision processing does not make use of vergence (i.e. using both cameras to determine
image depth) but does use both cameras in an effort to eliminate parallax. It does this by averaging the
final result for each – both the coordinates of the object and of the hand.
4.5.3 Object location
The colour of the object and the table were deliberately chosen so that the object could be easily discrim-
inated. This avoided becoming bogged down in image processing problems rather than an evaluation of
BOD. The method adopted was to scan the image bitmap for the right eye looking for areas with blue
higher than a given threshold, and red and green ones below the given threshold. The centroid is then
calculated by averaging the coordinates of each such point.The operation is then repeated for the other
camera and the mean taken of the coordinates of the object.
The technique proved very resilient, coping with a wide range of simulated table and object colours,
provided that the object was predominantly blue and the table lacked blue.
This very simple process is illustrated as follows:
(a) Original (b) Object isolated
Figure 4-5: Left camera image
4.5.4 Hand location and movement
The hand is visually guided to the cube by noting its positionand the position of the cube and moving
the hand so as to reduce the distance against the table. This has the advantage of eliminating the effect of
position errors but the disadvantage of requiring excellent visual hand location. Instead of only needing
to see the hand at two positions, the hand now needs to be identified and accurately located at successive
positions as it moves towards the cube.
It was this requirement which drove the replacement of the initial hand location mechanism with
a more resilient one. The initial one proved reasonably satisfactory when mapping from video frame
to kinematics frame was being used since it only required twohand locations. However it was very
simplistic and was not reliable. Furthermore it clearly could not work with the real robot.
To keep the image processing simple the hand is maintained ata single orientation whilst seeking
the object and the fingers are kept closed. As with ball location the position of the hand was determined
43
using the two cameras separately and averaging the positionto reduce parallax.
Noting the cube position and then moving the hand to that position was not an ideal approach from
a reactive planning perspective, but was relatively easy toimplement. It would have been ideal to
determine the object position each time that the hand position was about to be changed to compensate,
for example, for the object being moved closer or further from the hand by a human. However this was
ruled out since once the hand was very close to the object thenit would partially obscure it. Furthermore
the robot would need to process information about expected shape of the object as well as (or perhaps
instead of) its colour so that it could tell when part was covered up.
4.5.5 Initial algorithm
The first algorithm took a static image of the table with the object on it before the hand was moved to
the table. When the hand was moved to the table this static image was subtracted from the image of
the hand over the table. This eliminated most of the visual clutter and produced a very clean image. A
Canny edge detector (Canny, 1986) provided by OpenCV’scvCanny() function, was applied to pick
out the outline of the hand by looking for sharp well-defined changes in image.
(a) Original (b) Hand over table
Figure 4-6: Initial hand location algorithm: Images of the table without the hand and with the hand.This is before the change from a ball to a larger cube
Other filters were also tried, for example line detectors to try to locate straight lines formed by the
outline of the fingers, but the Canny filter proved to be by far the most successful.
(a) Subtracted images (b) Canny filter
Figure 4-7: Initial hand location algorithm: The two imagessubtracted to show differences between thescene and Canny filter applied
44
The final stage was to examine the processed image to precisely locate the hand. The process here
was to search across each raster line in the image looking fora particular sequence of black to white and
white to black transitions. The first scan line which contained that pattern was used as the Y position.
Once this was found the computer looked down each vertical raster line to determine the X coordinate.
(a) Line scans locate fixed point onhand
(b) Failure of the method – false posi-tive found
Figure 4-8: Initial hand location algorithm: Line scan locates fixed point on hand
This was reasonably resilient when the hand was in a couple ofset positions but was much less
reliable when it was used with object location using visual feedback as the hand moved around. It was
also clear that the method would not work with the more visually complex hand on the real robot.
The method typically failed either when some noise entered the image, for example sometimes part
of the object changes tone and so a small arc appeared in the differenced image, and was carried over
as a contour in the Canny filtered image. It also failed if too much of the arm was shown.
4.5.6 Replacement algorithm
(a) Original baseline image (b) Simulated hand over table
Figure 4-9: Replacement hand location algorithm: Images ofthe table without the hand and with thehand
The second algorithm used the same subtraction of an image without the hand present from one
with the hand present to reduce background clutter. It then used a sample image showing the hand as
a template and compared it in turn with each position on the subtracted image. It used the OpenCV
cvMatchTemplate() function to perform the matching. This results in a matrix containing a match
level for each point in the picture against which a match was done.
45
(a)Tem-plate
(b) Match of template against differ-ence image
Figure 4-10: Replacement hand location algorithm: Best match to template located and offset addedto precisely locate position of hand. Note that the cube shows up in the difference image, but does notaffect the technique.
(a) Cube has been replaced with an-other in a new position without a newbaseline image being taken. Hand notin view
(b) Cube moved slightly since baselineimage was taken
Figure 4-11: Replacement hand location algorithm: The algorithm is robust against the type of problemsseen using the simulator
This matrix was scanned for the highest value, and if above anempirically determined threshold the
corresponding point on the image was taken as the hand location. An offset was added to this position,
since it represented the top left hand corner of the matchingimage, to obtain a fixed point on the hand.
The method works because the code is looking for the closest match, not an exact match.
It also proved very resilient to the cube being moved or a small part of the hand being in view when
the baseline image was taken, and did not require modification when the task was extended to include
yellow cubes.
4.6 Inverse kinematics for limb movement
It was originally felt that the robot hand needed to move in two dimensions over a relatively small area
of table, and that inverse kinematics could be simplified by using visual guidance and small movements.
However the mechanical leverage and the large number of joints which need to be moved to precisely
position the hand meant that this did not turn out to be practical. Consequently at the outset inverse
kinematics was not considered important but it later developed into a significant amount of work.
46
4.6.1 What is Inverse Kinematics?
Kinematics is the mapping between joint angles and the resulting coordinates of the end effector, typi-
cally in Cartesian space. It is divided into two parts – forward kinematics is the method of calculating
the effector location from the position of joints and inverse kinematics calculates joint positions required
for a given end effector position. Forward kinematics is relatively straightforward – each joint position
and limb length joint can be represented by an array and computing the location of the end effector
is simply a matter of matrix multiplication. Inverse kinematics is much more complicated for several
reasons.
• Depending upon the design of the limb, it may not be possible to generate the solutions alge-
braically.
• There are impossible solutions. The desired location may simply be out of reach because the
limbs are not long enough; or the desired end effector rotation angle may not be possible to
achieve given the flexibility of the joints. For example it isnot possible to place your left hand in
front of your face, palm outwards and with the fingers pointing to the left.
• There are often several solutions if the limb has redundant joints. For example the hand may be
placed on a table and the elbow rotated about the shoulder so that it varies between touching the
body and sticking well out from the body. Different criteriamay be used to select a particular
solution, for example minimising joint movement or kineticenergy, or keep all joints away from
their limits.
• Collision avoidance can eliminate otherwise valid solutions, for example any inverse kinematic
solution which specifies that the elbow is inside the body is invalid.
In general six degrees of freedom are needed for a robot arm tomove to any position and any
orientation in space. The human arm, and thus the iCub robot arm, has 7 degrees of freedom, which
provide redundancy in possible kinematic solutions for many poses. However the restricted angle of
movement of some of the joints, in particular at the wrist, restrict some of the possible poses.
Inverse kinematics is widely considered a “solved” problemby many engineers, although it is still
the subject of considerable academic study. Most of the workon inverse kinematics dates from the
1960’s and early industrial robots and has given rise to the conventional engineering solution. This is to
start with the the forward kinematics equations for the robot, which provide the end effector pose from
the joint angles and use these to work out the joint angles needed for the end effector poses (see, for
example, Manseur, 2006, pages 151–183).
Some robot arms have ”closed-form” inverse kinematic solutions which can be solved algebraically.
The number, type and orientation of the joints is criticallyimportant for closed form solutions, and
most industrial robots are specifically designed to have arms of this form. It can be demonstrated, for
example, that a robot arm with 6 revolute (i.e. rotating) joints has a closed form solution if three of
the axes of rotation are either parallel to each other intersect (i.e. coincide) with each other. For this
reason many industrial robots having six degrees of freedom, to allow them to move to any position and
orientation, follow these rules. The commonly used “321 kinematic” structure, has three wrist joints
47
intersecting, effectively making a ball joint, a shoulder and elbow joint in parallel and a second shoulder
joint coincident with the first and at right angles it.
However more complex arms, including the seven degree of freedom human arm, are incapable of
algebraic solution and iterative methods must be used. The general technique is to use the forward
kinematic equations directly, estimating possibly joint angles and iteratively adjusting the estimated
joint angles the end effector position, and use this to iteratively adjust the estimate until the error between
the estimate and the actual position falls below a desired threshold.
There are other ways of resolving inverse kinematics, for example using lookup tables, and these
continue to be the subject of much research, in part motivated by a desire to understand how animals
perform inverse kinematics.
4.6.2 KDL
The first approach tried was to use KDL, the kinematics component of Ortocos, an open source robotics
project (Bruyninckx, 2001). KDL’s use with the iCub has beendescribed by Jesse (2009), but it proved
time consuming to implement, as the necessary documentation for the iCub proved inadequate, and there
was difficulty in getting the inverse kinematics iterationsto converge. This approach was abandoned
this approach through lack of time.
4.6.3 Other approaches
I was originally reluctant to use iKin, the iCub’s own kinematics library, which has replaced KDL within
the iCub project over the last few years. Whilst KDL uses Newton-Raphson approximations, iKin uses
IPOPT (Laird et al., 2006), an open source library for performing large-scale non-linear optimisation.
IPOPT in turn relies upon other software packages, and the one recommended by the RobotCub Con-
sortium is the Harwell Subroutine Library. In this mode IPOPT uses a pair of Fortran subroutines –
the double precision implementations of MA27 and MC19, which are used to solve symmetric matrix
systems. These routines are obsolete as far as the Harwell Subroutine Library are concerned and have
been moved to their “Archive”.
In principle it may be faster at reaching an inverse kinematic solution that KDL since it uses more
sophisticated algorithms, but the difference may be insignificant.
The Harwell Software Library and the Archive are commercialproducts although because its devel-
opment is partially funded by an EPSRC grant, they are available to academic institutions at no charge
provided that they are not used for commercial purposes. HSLhave also generously made the code in
the Archive to individuals for their own private, non-commercial use.
I felt that given the open source nature of the iCub project, and the code developed for this project,
it was better to use non-proprietary software software wherever possible.
Two other approaches were considered because of technical difficulties encountered with KDL and
IPOPT. Inverse kinematics is highly amenable to solution either using lookup tables or neural networks.
A considerable body of research has been done into both approaches with great success. Lookup tables
can be used by using learning techniques to build up the parameters for piecewise linear approximations
each covering a small volume (see for example D’Souza et al.,2001).
48
Neural networks have been widely used for inverse kinematiccalculations for twenty years, and the
number of papers probably exceeds those for other techniques by at least an order of magnitude. As
with lookup methods a particular set of weights only works for a small volume of space and a large
number of different sets must be used to cover the full volume.
4.6.4 iKin
iKin is very convenient to use since the kinematic chains describing the characteristics of the iCub
limbs are already defined in it. It also has code for handling one of the quirks in the iCub kinematic
specification, the connection between the torso and the arm.
The origin for the robot’s standard frame of reference is thetorso pitch joint. Thus to find the
position of the hand in the robot’s frame it is necessary to start at this joint and multiply in the matrices
representing the distances and orientations of the other two torso joints, the shoulder and arm joints.
Similarly for inverse kinematic calculations the torso joints can be factored into the joint angles needed
to achieve a particular pose for an arm. However since the torso joints are thus shared between the two
arms, they cannot be used simultaneously for poses of both arms since one may require, for example,
the robot to lean to the left whilst the other requires it to lean to the right.
Consequently the iKin library provides the ability to “lock” the torso (or any other joints) for any
particular kinematic chain so that their current angles areused as input, but are not changed. Thus in
moving two arms, the torso will be unlocked for one arm, and locked for the other. It also provides the
ability to restrict joint angles easily in inverse kinematic calculations – this is most often used to lock
the torso pitch joint so that new angles are not proposed for it, preventing the robot reaching forwards
too far and risk toppling with its stand. It will reach with its arm instead of bending over the table.
This feature was used since the robot bends over the table to move the shoulder forwards to extend
the possible reach of the right arm. The robot stands up straight when a new object is dropped onto the
table so that the object does not hit the robot before returning to its original pose. Even if it had not been
used in the initial task it would have been needed in the extension task since that uses both arms, and at
most one arm could use the torso for its kinematics.
4.6.4.1 MUMPS
Although a fully configured IPOPT library, which included the HSL subroutines (and much else includ-
ing METIS, a sparse matrix ordering system, for example) wasused for the early development work,
extensive testing and later development used IPOPT with just the public domain MUMPS linear solver
library (Amestoy et al., 2002), along with the free BLAS library (Basic Linear Algebra Subprograms),
which is required for all IPOPT configurations. No problems were found with MUMPS and there no
apparent differences in results to using the two HSL subroutines were seen.
In practice robotic inverse kinematics is at the very bottomend of the scale of problems that this
type of linear solver is designed to handle and MUMPS is perfectly adequate.
49
4.6.4.2 iKin problems
One aspect which iKin does not handle is finger positions – thekinematic model stops at the end of
the arm. The reason for this is that internally the inverse kinematics works on a serial chain of joints
– as soon as fingers are added the arm is no longer a chain since it is now terminated by a five leaf
tree. This did not cause any problems but would need to have been considered if path planning had
been implemented since iKin would need to be used to ensure that the fingers did not hit the table. The
solution considered for path planning was always keep the fingers in the same position and consider the
hand as a rectangular cuboid of known size which could not intersect with the table or other objects.
iKin sometimes complains about infeasible poses when askedto compute joint settings for poses
which have been successfully calculated previously. To mitigate these problems, the program repeats
the calculation up to five times if there is an error before raising an exception.
Various steps are taken in the program code to ensure that joints are never specified outside of
permitted ranges – the joint angle corrected to fall at the limit if just outside as a result of rounding or
sensor error.
Since the human arm has seven degrees of freedom there is considerable redundancy in most poses,
except at the limits of reach. I was largely unconcerned withthe particular joint angles needed to achieve
some performance and so used the default weighting in the iKin algorithms to choose one redundant
set of joint positions over another. If path planning had been implemented then this could have been an
issue, since one pose might put the elbow through the table when another pose did not. Real humans
have additional criteria over anthropomorphic robots, which are to minimise the amount of energy
expended in adopting the pose, and to maximise comfort by notraising the arms above the head and to
avoid moving certain joints, notably in the wrist, to the extreme of its limits.
4.7 Avoiding obstacles
A constant source of problems was the robot’s hand striking the table. This usually happened when
the robot was instructed to move from one pose to another. Theunderlying code simply instructs the
different joints to move simultaneously to the necessary angles to achieve the new pose. It was often the
case that the hand struck the table since no path planning hadbeen included in the program.
This was mitigated by careful choice of poses – in particularmaintaining the robot’s hand high above
the table whenever possible and ensuring that its fingers were aligned parallel to the table whenever the
hand was close to it.
The cube is deliberatively dropped at random positions on the table, but the robot’s field of view
is sufficiently small that the suitable area of the table to beused only corresponded to a few square
centimetres. If a larger expanse of table were used then it ishighly likely that path planning would be
needed.
Considerable thought was given to adding path planning, butit would have been a major undertak-
ing. This would have taken the form of extending the kinematic model to include the fingertips with
the hand in the open position. The fingers are not included in the iKin kinematic model because it is
designed to handle chains without branches, and so conceptually it would have been a further five chains
50
added to each arm. However the fingers could be treated as a single link in the chain if the fingers are
straight and together. A path would then have been worked outchecking that a volume around each
limb, including the hand up to the finger tips tips did not intersect with the table, the cube, or the robot’s
body. If a problem was discovered then the path would need to be adjusted to avoid the problem, which
in turn brings out problems of inverse kinematic points which are not possible to reach.
Path planning was also one of the issues before inverse kinematics was introduced, since it became
clear that small changes in the arm position, especially when the hand was close to the table, would result
in the hand hitting the table or the cube. Forward kinematicsquickly became important to predict when
this would happen (although perhaps the touch sensors mighthave been used instead). However once
the principle of inverse kinematics was introduced then there were now conceptually small movements
which were detected as impermissible because they would strike the table or the cube.
4.8 Grasping and picking up the object
Grasping was far and away the hardest part of the work, and well over a week was spent experiment-
ing with different techniques; none of which proved successful with the simulator. It was important
to implement because it is an example of precise control of low level functionality requiring precise
coordination between different parts of the robot.
The simulator is very simplistic as regards contact betweensurfaces, with frictionless joints and
no rolling friction. Consequently grasping proved enormously difficult, to the point where serious
consideration was given to abandoning either grasping or the simulator. However the lack of availability
of a real robot meant that grasping had to be implemented evenat a very simplistic level.
The simulator differs in this respect from the real robot. The real iCub has a single tendon controlling
the opening of each joint (or each joint direction since the thumb can be manipulated in three directions)
in each finger. Closing is done by a spring pulling against thetendon. Thus to close a finger against an
object all that needs to be done is to relax the tendon past thepoint where the finger touches the object
and a lateral force will be applied by the spring. If a finger orthumb is simultaneously being applied
to the opposite side of the object then the result will be pressure between the fingertip and the object
leading to frictional forces.
Numerous approaches were tried using the simulator. The prototype grasping code was originally
implemented in C++ but when serious problems became apparent it was moved to the Python code to
facilitate more rapid prototyping of possible solutions. The use of Python for the highest level control
in the code proved particularly useful for this since it was far faster to prototype solutions using it than
when the code was in C++.
All of them started by lowering the hand onto the cube until the touch sensor on the palm was
triggered and then raising the hand slightly. In this way thefingers would close onto the cube rather
than hitting the table.
51
4.8.1 Thumb/index finger pincer
One approach was to place the thumb and index finger opposite each other and close them until the joint
velocity indicated that both had contacted the object. The lack of friction usually meant that the object
slid out of the grasp. This was particularly troublesome with a ball, since the ball could easily be set
rolling by the slightest touch. The approach was modified to gradually close all of the fingers until each
one stopped as it touched the object.
This is how the real iCub robot code is implemented – all digits on the joints are set to close at
a constant velocity and explicitly stopped when their velocity falls to zero. It is necessary to use this
rather than the fingertip touch sensors as these sensors are only optionally fitted to robots.
Like the simulator, the real robot lacks any force control inclosing the fingers, since the opening of
a finger is done by shortening a tendon, whilst the closing is done using a return spring. In contrast, a
human has a pair of opposing tendons to straighten and bend each finger.
Thus the amount of force which can be applied by the iCub fingeris within a very limited range
since the spring is weak. This is not a problem operationallysince the robot is normally required to
grasp a reasonably light, rugged and usually deformable object which does not require precise force
control.
However in the simulator, this approach was no more successful that the original approach using the
simulator since the object still slid out of the way.
4.8.2 A cube instead of a ball
Initially a ball was selected as the object to pick up, since the symmetry would make the orientation
of the object irrelevant. This could be particularly important in the extension to the original task, for
example if the object was to be caught. However when trying toimplement grasping I discovered that
the simulated ball would start rolling at the slightest touch. After struggling to prevent this I replaced
the ball with a cube. Experiments were done to optimise the size of the cube – too small and the cube
would easily slip through the fingers if the hand were not precisely positioned; too large and the hand
would not grasp the cube – a compromise size of 5 cm was settledon.
4.8.3 Emulating the robot
Another approach tried was to incrementally move the fingersby a very small angle until the touch
sensors activated and then close the fingers slightly more totry to apply pressure to the object. This
was unsuccessful since the simulator (and the real robot) blocks until the joint has moved to the desired
position. This was modified by adding a timeout aborting the motion after a short interval, but did not
prove any more successful in picking an object up.
4.8.4 Thumb/little fingers pincer
The most successful approach was to use the fingertip touch sensors and set all fingers to close, halting
each finger as the touch sensor was activated. The thumb was set to a position where it would be in
opposition to the ring and little fingers (which are driven asa single unit in the iCub), this being more
52
Figure 4-12: The simulated robot some-times successfully picks up the cubewithout using the simulator’s “grab” fea-ture, as here, but frequently drops it as thehand is moved. The cube has just fallenout of the hand
successful than driving against the index or any other finger. Closing the first two fingers adds little to
the success of this method. If the touch sensor deactivated again because the cube had moved then the
finger was set in motion again until the touch sensor activated. Using this approach it was possible to
pick up the cube roughly ten percent of the time, although it would frequently fall from the hand when
it was lifted a few cm from the table.
Various techniques were tried to mitigate this, such as raising the hand very slowly and twisting
it so that gravity was no longer operating to pull the cube from the fingertips, but none of these were
consistently successful.
Other approaches were experimented with, such as sweeping the cube off of the table with one
hand into the outstretched palm of the other hand, but they required too much complexity in the robots
movements. For example to sweep the cube into a second hand then visual tracking of the second hand
would be needed along with additional code to determine the table thickness so that the second hand
could be placed just underneath the table so that the fingers did not hit the edge of the table. Fine control
of the cube position would also be needed, along with route planning to ensure that the sweeping arm
did not hit the top of the table.
4.8.5 Simulator “grab”
To avoid these problems the simulator provides a “grab” operation which creates an invisible rigid link
between the hand and the object to be picked up. Thus the object becomes an extension to the hand. I
was not keen on using this partly because it meant that the simulator code would not reflect code for the
real robot, and partly because it does not require the robot hand to approach the object to be picked up
at all. The connection between hand and object can even be made with the robot arm at its side.
It was decided that a compromise would be made, to grasp the object in the most successful way
possible and then enable the “grab magnet”. Unfortunately the grab functionality is explicitly disabled
in the simulator when finger control is enabled, so it was necessary to modify the simulator to permit
this to be used. The change required was not great, merely to disable the check on whether finger control
is enabled. Internally the object to be grasped is connectedto a simple hand object when the fingers are
disabled and to the end of the palm where the fingers are connected but this code already existed in the
simulator.
The modification is simple to apply to the simulator, or thereis a fork of the simulator, which fixes
53
Figure 4-13: Testing. The robot failedto pick up a cube within the timeout in-terval. The POSH-driven recovery is tochange the cube colour and move it toground level. The cause was eventuallytraced to the failure of a mechanism in abehaviour which prevented the hand frombecoming locked into a cycle of over-shooting the cube and then undershootingit again
some of the other problems in it, which forms the SourceforgeSimforkproject. This version of the
simulator is completely compatible with commands to the existing version.
4.8.6 Other approaches to grasping
Some limited experiments have been tried using the modifiedSimforksimulator, notably by adding
damping to bodies, without resolving the problems of grasping in the simulator. In the simulator damp-
ing is used to reduce the velocity of two objects in contact. The simulator allows bodies to be provided
with damping, which comes into effect above a critical velocity and applies constant a force in the op-
posite direction to the direction of motion. This is simplerthan the friction implementation, which is
specified for pairs of bodies.
4.9 Testing
The code was tested by limited unit testing of the lower levelcontrol classes but mainly carefully
observing the operation of the full program and studying thelog files. This was appropriate since
the goal of the work was to evaluate BOD rather than to producea highly refined set of underlying
control classes.
The “standard” regression test was to have a separate dedicated computer running the code, handling
around 90 cubes in each run and any problems thrown up examined. The limit of 90 was caused by the
slow-down of the code caused by the extra work the CPU performs in rendering the two piles of cubes
on the ground, and performing physics calculations on them.Further work in the Simfork simulator
may have alleviated this slow-down but it did not impact the work described in this thesis.
Developing the high level plan using BOD is an iterative process, where the code is gradually built.
I started with all actions and senses implemented as dummy functions and gradually filled them in with
functional code, frequently running the code to observe thenew behaviour.
The only realistic ways at present of testing the plan is by running it repeatedly or by static analy-
sis. Testing the behaviours and the layers below the behaviours was much more straightforward, since
conventional testing methods can be used. Because they are much more modular they could be tested
in isolation from each other. Given more time (and a different emphasis for the project), it would have
been possible to modify the simulator to store states to allow a full set of automated regression tests to
54
have been developed. The simulator would be loaded with a particular state that the reactive plan would
respond to in a particular way. Any code modification could have been automatically tested with several
dozen states to ensure that it did not break anything.
4.9.1 Obsessive behaviour
One particular problem encountered during software testing was automatically detecting and handling
obsessive behaviour in the robot.
This type of behaviour occurs when the robot just fails in carrying out a task, and enters a loop
where it constantly repeats the sequence of actions, just failing each time. Note that this is different to
“dithering”, in which a robot oscillates between two different behaviours as each are triggered in turn
repeatedly. The problem seen here is that robot carries out asequence of actions only to find at the end
of them that its goal in doing so has not been satisfied.
For example in the competence sequence below, the iCub will ensure that its hand is open, then
move the hand to be over the table if it is not already over the table. If it cannot see the hand then
it moves the hand to a position close to its waist where it can (the “start line”). It then progressively
moves its hand until the hand is over the cube. However if it has problems because, for example the
hand moves out of view since the cube can be located more easily than the hand around the edges of
the robot’s field of view, then the robot will return the hand to the start line on the next pass through the
competence.
(C pick−up−cube
(MINUTES 1)
(GOAL ( ( h o l d i n g c u b e ) ) )
(ELEMENTS
(
( open−hand (TRIGGER ( ( handopen 0 ) ) ) openhand )
( move−hand−to−table (TRIGGER ( ( h a n do v e r t a b l e 0 ) ) ) m o v e h a n d t o t a b l e )
( move−hand− to−start− l ine (TRIGGER ( ( h a n d v i s i b l e 0 ) ) ) m o v e h a n d t o s t a r t l i n e )
( move−hand−to−cube (TRIGGER ( ( handove r cube 0 ) ) ) movehand to cube )
( pick−up−cube (TRIGGER ( ( handove r cube 1 ) ) ) pick−up−cube−ap )
)
)
)
The movehandto cube behaviour is thus generating an error which cannot easily be handled. The
solution was to request the cube be removed and another one dropped onto the table within the robot’s
field of view. The robot then scans the table for the new cube and updates its cube position.
To solve this, a “failed” flag was set in movehandto cube method, which is interrogated as a higher
priority goal in the main set of drives. Although there is a variable called “failed” which is now set or
examined by a method called from different parts of the code,it did prove remarkably easy to fit this
into the POSH plan.
There is also a catchall to trap other types of errors of this nature; a timeout that removes the cube
and resets the robot to an initial state if the cube is not picked up within two minutes. This timeout
interval is much longer than the robot normally takes to pickup a cube, about 15 seconds. POSH itself
55
also provides timeouts but these work slightly differently; if an action is on the scheduler stack for
longer than the timeout interval then they are removed. However this feature was not used since it does
not send a signal so that the robot environment can be reset.
4.10 Extending the original task
The second stage of the project was to extend the initial implementation to evaluate how well BOD
was in coping with changes. The second step had deliberatelynot been considered to any great degree
during the first step.
The new task was to be an extension of the original to add significant new functionality to the pro-
gram. It needed to make use of the original task but be sufficiently large to possibly require refactoring
of the original code.
The new task was to modify the program so that a random mixtureof blue and yellow cubes would
be provided to the robot. Blue cubes would be dealt with as before – picked up by the right hand and
discarded at arms length on the right side of the robot’s body, but yellow cubes would be transferred to
the robot’s left hand and discarded at arms length on the leftside of the robot.
The principal area where the program would require refactoring was in the new decision process
related the cube colour and the mechanics of transferring the cube from one hand to the other, which
would require code for the right arm to be modified and code to control the left arm introduced.
4.10.1 Modifications to C++ control classes
The following changes were identified in the underlying C++ class library to implement the next func-
tionality:
• Modifying the World class to drop cubes which were either randomly yellow or blue was trivial.
• The vision processing software was easily modified to searchfor a yellow cube if the search for a
blue cube failed.
• The visual hand location processing software did not require any changes.
• The only other change needed was to convert some of the classes to singletons, since they make
network connections to the simulated iCub subsystems and these cannot support more than one
simultaneous network connection.
4.10.2 Behaviour modifications
The original code had been implemented as a single behaviour, corresponding to a single Python class
which provided methods to move the hand, move the robot torso, search for the cube and grasp. This
is not generally recommended for BOD but the task was considered sufficiently simple that there was
no need to start with separate behaviour classes for these, and it would be easier to combine different
features – for example if vision needed to directly interactwith the robot hand, perhaps spreading the
fingers or changing the angle.
56
(a) Robot picks up cube as before (b) Since it is a yellow cube it elevatesit so that it can transfer it to its left hand
(c) Places left hand under right handand cups left hand
(d) Right hand drops cube into left hand (e) Withdraws right hand (f) Slowly moves left hand to left of ta-ble
(g) Extends arm (h) Drops cube on ground (i) Returns to initial position
(j) Clears table and requests anothercube
(k) Blue cubes dropped from right handas before
Figure 4-14: The extended task – sort blue and yellow cubes
However this deviation from the recommendation in the methodology turned out to be a mistake
since it made extending the task harder. At this point it became clear that the existing behaviour needed
to be split into several separate behaviours to make them more manageable.
The behaviours were split along the lines proposed as part ofthe BOD methodology, by looking at
state variables rather than, for example common blocks of functionality, such as all actions involving
the right arm or vision processing.
This approach worked extremely well, and should certainly be adopted in any future refactoring
57
of BOD software. Not surprisingly the behaviours did not correspond to the underlying C++ classes,
which were concerned with functional modules such as arms and legs.
Slightly more formally the steps were:
• Identify sub-behaviours based on state attributes
• Split out the first sub-behaviour my moving it and its states to a new class
• Test the system with the modified behaviour
• Repeat for next behaviour
Slightly surprisingly it was necessary to make some small changes to the POSH plan, as functional-
ity which was previously implemented by a single method was split between two methods in different
behaviour classes.
Some interaction between behaviours was needed, for example if picking up the cube is aborted
because the robot has detected that the fingers have not grasped within a certain time, then the cube
position has to be flagged as unknown since the cube can be nudged as it is grasped, which requires the
robot to relocate its position. This was accepted as inevitable.
The final set of split behaviours were as follows:
• FindingTable – measuring the table height
• HandlingErrors – checking timeout and fail flags
• LocatingCube – visually locating the cube on the table
• PickingUpCube – moving the right hand to the cube and picking it up
• RemovingYellowCube – this is the code which handles the transfer of the cube and its re-
moval
• RightHandGrasping – grasping with the right hand
4.11 Description of the behaviours
4.11.1 FindingTable
This behaviour is concerned with finding the table height, although it also records an x and y position
for the table. Thereafter is is responsible for moving the right hand when it needs to be moved with
respect to the table, for example out of view when the robot islooking at the table. The robot places
its hand over where the table is expected to be and slowly lowers it until one of the touch sensors in
the hand is triggered, at which time it raises its hand again having recorded the coordinates. The status
element around which it built is the table coordinates and whether they have been determined yet or not.
The table coordinates are accessed read only by several other behaviours.
58
Action/Sense primitive Description
a find table Finds the position of the table by placing hand over it,
gradually lowering it until it touches the table and with-
drawing hand again
a move rhand above table Moves the right hand over the table in such a way that it
does not strike the table
a move rhand clear Move the right hand clear of the table so that the robot can
see the table without the hand being in view anywhere
a move rhand to table Moves the right hand to the table, at a height above it
where it will skim over cube
s hand over table Sense determines if the right hand is anywhere over the
table
s table found Sense returns flag indicating if the table has been found
4.11.2 LocatingCube
This behaviour is responsible for locating the position of the cube on the table. The state element
underlying it is the cube position.
Primitive Description
a find cube Locate the cube visually on the table
a lose cube Forget the location of the cube on the table
a request new cube Moves robot body clear and requests a new cube be
dropped onto the table before restoring robot
a request remove cube Request the cube on the table be removed
s blue cube Determines if the last cube seen by the robot was blue
s cube absent Determines if there is no cube on the table
s cube found Determines if the cube position on the table is known
4.11.3 PickingUpCube
This behaviour is responsible for picking up the cube in the right hand, and is built around the hand
location which is determined whenever needed by the Vision system. It uses the cube location provided
by the LocatingCube behaviour.
59
Primitive Description
a home Move the robot to a known state, with hands by side. This
primitive doesn’t really fit comfortably in any behaviour
but giving it a behaviour of its own would be excessive
a move rhand to cube Moved the hand towards the cube. If the hand was close
to the cube the increment was reduced. Originally this
worked as a loop, but was split out into a action triggered
multiple times from a competence element.
a move rhand to start line Move the hand to a position over the table where it can
be seen and is sufficiently off the table that it will not hit
the cube
a remove cube Lift up the cube in the right hand and discard it
s rhand clear Determine if the hand is clear of the table
s rhand over cube Determine if the hand is over the cube such that the cube
can be grasped
s rhand touch Determine if the hand is touching anything
s rhand visible Determine if the hand is visible
s sufficient cubes processedCheck if enough cubes have been processed to meet the
drive collection goal
4.11.4 RightHandGrasping
This behaviour is responsible for grasping the cube once PickingUpCube has moved the hand to the
correct position, and once the cube has been grasped, PickingUpCube lifts it from the table. It is built
around state elements which hold the status of the grasping process – whether it is in progress and
whether the cube has been successfully grasped. The left hand does not grasp the cube, the cube simply
rests in the palm and so this behaviour is specialised for right hand grasping.
60
Primitive Description
a abort rhand grasp Abort grasping: stop fingers closing
a close rhand Action configures the right hand for grasping. It sets the
fingers moving to close around the cube
a open rhand Open hand
a raise cube Action lifts the cube off of the table, assuming that the
right hand is holding it.
a release cube Release simulator magnet grab on cube
a set rhand height Lower hand until the palm touches something – assumed
to be the top of the cube, then raise hand by a small
amount so it is just above cube.
a update rhand grasp Action starts and stops fingers whilst closing them around
cube. Finger stopped if touching something, started clos-
ing again if not. When all five digits are touching cube
it has been grasped, unless simulator ”magnet” is used
when we don’t really need to grasp the cube, just make
an effort.
s fingers set for grasp Sense indicates whether the fingers have been configured
to start grasping
s rhand grasp sanity check okSense checks if the right hand has been grasping for too
long – cube probably pushed out of the way
s rhand height good Sense indicates if the right hand has had its height above
the cube adjusted for grasping
s rhand holding cube Sense indicates whether the right hand is holding the cube
s rhand open Sense indicates if right hand is open. Look for index fin-
ger being reasonably straight
4.11.5 RemovingYellowCube
This behaviour is responsible for the left hand, and the processing of the yellow cube once the PickingUpCube
behaviour has picked it up in the right hand. The PickingUpCube behaviour is responsible for the right
hand during the transfer to the left hand, whilst this behaviour handles the left hand.
61
Primitive Description
a cup lhand Make the left hand into a cup to receive the cube dropped
into it
a grasp lhand Grasp the cube in the left hand. The fingers are not fully
closed around the cube as this risks squashing the cube
out of the hand but the hand is tilted so that the cube tends
to roll towards the fingers.
a move lhand in Raise the right hand, which is assumed to be palm down-
wards and holding the cube, then bring the left hand in
underneath it
a remove yellow cube Remove the yellow cube by swivelling the body to the
left. This is done in very small steps because the cube is
not firmly held, and any great movement tends to make it
fall out of the hand. When it has rotated drop the cube.
This primitive does not do this in large steps but incre-
mentally in small steps so that the plan remains respon-
sive.
s holding cube Sense indicates whether the iCub is holding the cube in
either hand, used to control the transfer from the right
hand to the left hand. It isn’t that straightforward since
during the moment of transfer neither hand is holding the
cube, so if the left hand is attempting to hold the cube
then this also counts as ”holding”
s lhand grasping Return True if the left hand is in the process of grasping
the cube
s lhand holding cube Sense indicates whether iCub is holding the cube
s yellow cube disposed of Return True if the yellow cube has been dropped on the
ground
4.11.6 HandlingErrors
This behaviour is responsible for error handling, and its state elements are the timeout and the error flag.
It does very little, simply being responsible for maintaining these variables. Other behaviours reset the
robot back to its initial state and request the cube to be removed and replaced with another one.
62
Primitive Description
a reset fail Reset the fail flag
a reset timeout Reset the timeout flag
a set fail Set the fail flag
s failed Obtain the value of the failed flag
s succeed Always return true. Isn’t really part of error handling but
fits in this behaviour better than elsewhere
s timeout Obtain the value of the timeout flag
4.12 Naive users
Two individuals, each with many years professional software development experience but who had no
exposure to robotics or reactive planning agreed to undertake a small extension to the software. The
exercise was undertaken to understand how easy people new toBOD would find the methodology and
action planning. The exercise took around 2.5 hours per person. The purpose of the exercise was to
confirm, or refute, my own observations in learning BOD.
4.12.1 Task design
The task given was to drop the blue cube immediately after picking it up of it had not been handled
before. The robot would then pick it up again and dispose of it. The robot simulation running the
extended task was provided fully installed and operationaland they were talked through how it worked.
The task was selected as being easy to understand and capableof being implemented within a short
time with few changes to the existing behaviours. No new actions involving movement of the robot
were required, since these proved time consuming to debug inthe initial implementation, but it would
require understanding and extending the logic behind the plan and implementing a new state variable.
Reactive planning, Basic Reactive Plans and behaviours were explained and during the program-
ming they were helped with the syntax of the .lap files and Python programming language, and notes
were taken of their comments and actions. They each answereda short questionnaire at the end of the
exercise.
At the end of the task each user was asked a set of questions:
• Did you feel that reactive planning made the task easier or harder?
• Did you feel that the methodology made the task easier or harder?
• What did you feel that the syntax of the reactive plan helped orhindered?
• Do you have any suggestions about how to make the task easier to implement?
• Did you find anything particularly beneficial?
• Did you find anything particularly troublesome?
• Would you consider using it in some other project?
63
• Have you any other comments?
4.12.2 First user
The first user thought the syntax of the .lap file was clear, andimmediately commented on how easy it
would be to have a sequence moving the hand to the left and another moving the hand to the right so
that the robot just oscillated between the two.
The task, as he specified it, involved:
• Creating a new state, recording whether the cube had been handled before
• A sense to obtain its value and actions to clear and set it
• Clearing the value when a new cube was requested
• Setting the value when the cube was dropped the first time
• A new action pattern to drop the cube the first time it is pickedup, and setting the “handled flag”
in it
• A new competence
He experimented with putting the new competence to check thestate and drop it in several places,
and commented on how easy it was to get the task to hang becausethe robot was checking the flag and
attempting to drop the cube before it had picked it up. The competence was
( drop−cube− i f −unhandled (TRIGGER ( ( sh a n d l e d c u b e 0 ) ) drop−cube ) )
The jyPOSH logging proved useful when the syntax had been explained to him.
Four comments stand out
• “The law of unintended consequences is really abundantly clear”
• “Very easy to get it into a dithering state”
• “Where it looks like a sequence but is not a sequence is very deceptive”
• “Everything seems designed to slow you down”
Overall he picked up the syntax very quickly but was disappointed about how hard it was to carry
out a task.
4.12.3 Second user
The second user also understood how POSH worked without any difficulty and immediately indicated
the need for a new state variable, the number of times the cubehad been handled. The variable was
initially a boolean, but modified it to a counter as it needed three values for the number of times the
cube had been handled - 0, 1 and 2.
He solved the problem differently from the first user, creating a new competence for handling the
blue cube,
64
(C check−new−cube (GOAL ( ( s new cube 2 ) ) )
(ELEMENTS
(
( new cube (TRIGGER ( ( snew cube 0 ) ) ) drop−cube )
( o l d c u b e ( T r i g g e r ( ( snew cube 1 ) ) ) dispose−of−blue−cube )
)
)
)
wheredrop-cube was a new action pattern based on anddispose-of-blue-cube was the
original pattern for disposing of blue cubes.
He understood the purpose of reactive planning and saw how itcould be used to add flexibility to
handle unexpected events. He felt that the syntax was not good, particularly of the Python, and that
programming should be via a GUI.
65
Chapter 5
Results
5.1 Evaluation of the BOD methodology
The BOD methodology worked very well and was easy to apply to the problem. The early generation
of the list and action primitives and resulting POSH plan that was incrementally filled in was highly
successful. It meant that the system could be built and tested incrementally, and the high level design
was never so complicated that it was hard to rework it. It alsoprovided a good focus for understanding
the problem.
It proved a good way of making quite a complicated problem manageable by splitting it into lit-
tle bits. It also allowed minimal functionality to be reasonably simply implemented and refined and
expanded upon later. The method allowed quite large changesto be made, probably more than more
conventional object oriented programming would allow since the behaviours were more or less stand
alone. It rather blurred the clean design of a class since it was viewed as a collection of behaviours
which may not have too much relationship with each other.
However whilst BOD did prove very successful it was at times hard to determine the the boundaries
of the methodology and whether I had strayed outside of it. This is further explored in Section 6.1.
5.2 POSH
POSH plans were very flexible and easily reworked to change the behaviour of the robot. They proved
resilient, since the failure of the robot to achieve some goal, for example moving the hand to the cube,
would be corrected when the goal was once again addressed. One perverse problem was that it made
it harder to find bugs – at one time during the development the hand was moved in a loop until it was
close to the cube, and could exit early from the loop if an error occurred. The POSH plan would call
the method again when it found that the cube was not yet near tocube. One perverse difficulty was this
resilience made debugging, in this case finding and removingthe exit, hard. I spent about an hour trying
to work out why code at the end of the method was only sometimescalled.
66
5.3 Tools
5.3.1 jyPOSH
jyPOSH was easy to use although it was not clear how the timeouts worked and with which type of drive
collection because of poor documentation. The examples proved useful, but do not show subtleties and
features not used by those examples.
The plans are easy to write in the .lap format although there seem to be rather too many nested
brackets in some of the constructions. Rules on naming wouldbe useful, for example attaching the
behaviour name to each action and marking action pattern names and competences. De facto rules can
be implemented, but some standardisation, possibly enforced by the engine, would make navigation
around the actions easier.
In general though, provided names are chosen sensibly, the .lap plans give a concise and effective
overview of the program, and serve to provide a simple high level document describing the whole system
– something that is often lacking in more widely used software methodologies.
A potential problem with jyPOSH is that it is not possible to reuse part of a plan with a different
behaviour except by cutting and pasting the code. An examplemight have been to have re-used the
right hand grasping competence with the left hand. In jyPOSHthis can only be done by modifying the
grasping behaviour so that it will work with either hand so that both types of grasping are implemented
as a single behaviour.
5.3.2 ABODE
The Abode editor was used to regenerate the POSH plan experimentally. The editor was instinctive to
use but had a few minor bugs. Having spent some time working onthe POSH plan manually it felt
rather clumsy. This is a common criticism of simple GUIs, that they work well for people unused to
the underlying file format but not for people who are familiarwith it. The GUI is really an editing tool
– if it provided other features such as some sort of conditional breakpoints then it would have been
considerably more useful.
In general it was more useful as a syntax checker and diagramming tool than a plan editor. Never-
theless Abode would be a useful tool for teaching the mechanics of POSH Basic Reactive Plans.
The bugs found were relatively minor:
• The drive collection type cannot be changed easily. It is created as a non-real time type. If the
drive collection is deleted and recreated then it is createdas a real-time type.
• The output .lap file has Windows format line endings on Linux.
• On a double headed monitor, if the main ABODE window is moved to an ancillary monitor, the
pop-ups are still displayed on the first monitor.
• The program crashed if given a .lap file missing timeout fields. It apparently expects the input file
to be in the same format with the same optional parameters as it produces. A better input parser
is needed if it is to process plans which have been manually generated.
67
• The editor crashes if presented with drive patterns, competences or action patterns which do not
have timeouts in them.
5.4 Naive user experiences
The experience with the two naive users indicated that the BOD methodology, strict POSH and the
jyPOSH .lap file syntax, is easy to pick up, which confirms my own experiences. I would also echo
the comments about the difficulty in predicting behaviour without experimenting are apt – “unintended
consequences” – and some better tools for developing the plans and exploring the effects of changes to
the plan would be beneficial.
5.5 Simulator problems
The use of the simulator highlighted several important differences from the real robot.
5.5.1 Poor fidelity
Features of the actual robot, such as errors in the joint positioning sensors, whiplash in the camera
motors are not simulated. This means that many techniques which work with the simulator will not
work with the robot and vice versa. For example the simulatorwill precisely place the robot’s hand in a
particular position, but on a real robot errors are likely tomake this less accurate.
5.5.2 Forces
The simulated robot has no ability to apply forces. This is important for grasping. The real robot has
a wire “tendon” attached to each finger and operating againsta spring. A motor shortens the tendon
causing the finger to open. If the tendon is lengthened then the finger closes under the force of the
spring. Consequently in grasping as the robot closes its fingers around an object the fingers apply the
force of the spring to the object. The simulator has no equivalent finger tip force, so that the fingers
close around an object by moving to a position. They cannot intersect with the object so stop at its
boundary but do not apply any lateral force.
5.5.3 Sensors
It also lacks any modelling of force sensing, which has been included in the later versions of the robot.
In the real robot this takes the form of torque sensors in the hubs of some of the motors, and sensors
built into the upper arm and thigh. The simulator does feature touch sensors in the fingertips and palm
which to some extent mitigates this since contact with an object can be detected.
Touch sensitive fingertips have been developed (see Schmitzet al., 2008)), but are as yet installed
on very few of the robots. They are much more sophisticated than the simulator’s on/off sensors. Each
fingertip has 12 sensors which use capacitance to measure theseparation between conductive layers
68
applied to the outer surfaces of a compressive foam coveringthe fingertip, and consequently provide
quantitative pressure measurement.
5.5.4 Simulated grasp
The simulator has a mode whereby it can simulate grasping. This is only available in the mode where
finger-less hands are selected. It operates by making an invisible rigid link in the underlying ODE model
between the hand and the object being picked up. Consequently there will be a rigid link between the
hand and the object, no matter what the distance between the two is.
69
Chapter 6
Discussion and analysis of the results
6.1 BOD
The BOD methodology was very successful in producing the task. The key points were that it allowed
the task to be developed iteratively until it was complete and worked, and split the problem into man-
ageable parts.
In the implementation of the task, the amount of effort on BODwas rather overshadowed by the
effort required in the graphics and the kinematics – this should be regarded as an advantage since the
converse would have indicated it was not really suitable.
However whilst BOD did prove very successful it was at times hard to determine the limits as a
methodology. I frequently wondered “is this BOD?” and whether I had strayed outside of the method-
ology when I was making design decisions. A case in point is the implementation of grasping as afor
loop in a program. It caused an action to take several seconds, so prevented any higher priority goal
from being triggered for that time in Strict POSH. Whilst it iseasy to follow the guidelines of BOD
it was not at all clear what is not correct. Another case is thedecision to write the first version of the
code as one behaviour – whilst this is not recommended, it is not “disallowed”. The problem, at least
for evaluating it, is that BOD is too permissive – it gives theimpression that anything is allowed. For
example, a behaviourshouldbe an object but doesn’thaveto be. For instance:
in a BOD agent,how is controlled by a set of modular programs, calledbehaviors.
[Bryson (2001a, page 25)]
The way we sayhow under BOD is using object-oriented programming methodologies.
in BOD we decomposehow into modules calledbehaviors, which we code as objects.
[Bryson (Both 2001a, page 27)]
2.2.5 Behaviors that aren’t objects.
[Bryson (2001a, page 30)]
70
Bryson is really saying that behaviours can be implemented in any number of ways, of which objects
are the most natural but other techniques can be used if they are more appropriate for the task at hand,
or if the implementation language makes coding as objects unsuitable. Coding it as an object, or as
something else is neither right or wrong, but a decision based upon expediency. Whilst this flexibility is
sensible it does make defining the borders of the methodology, pinning down what is within the bounds
of BOD and what is just outside of it, rather difficult.
In a practical project this might just be an irrelevant and abstract question of semantics; that a
technique used is part of BOD or is not does not matter if the goal is a useable, working system. But in
evaluating the methodology the borders need to be clearly defined otherwise it is not clearwhat is being
evaluated.
6.2 Behaviours as objects
In practice behaviours in BOD will be objects, or at least instances of abstract data types, however they
are implemented, provided that the BOD methodology is followed since they represent a state and the
primitives for manipulating that state.
An object is an entity that has a state and a defined set of operations that operate on that
state.
[Sommerville (2006, page 316)]
However although the actions and senses are implemented as methods, and affect the state of the
object, the behaviour does not primarily provide methods for other objects (i.e. other behaviours) to
use. In a conventional object oriented system, the objects interact with each other, and the operations in
one object are providing services to those other objects. However in BOD there is comparatively little
interaction between behaviours since the behaviours interact primarily with the POSH engine. Where
there is interaction this tends to follow the object oriented paradigm – for example theFindingTable
behaviour provides the table height to other behaviours which need it; but in the main behaviours are
not used in an “object oriented” way even though they are objects.
Underneath behaviours there are objects which are used in a much more conventional way and really
do provide services to the behaviours. The consequence of this is that behaviours cut across several
different areas of functionality in a way that most classes in conventional object oriented programming
do not, so that several different behaviours may use the sameunderlying classes in the same sorts of
ways, for example commanding the robot’s arm to move to a particular position.
In some ways this is reminiscent of Aspect Oriented SoftwareDesign (Elrad et al., 2001) – be-
haviours look quite like AOSD’s cross-cutting concerns andthe classes underneath them like the core
concerns. The details are rather different though, there isno concept of a weaver assembling the be-
haviours from different source files, and it would be hard to see how such a tool would actually be useful
in BOD. Nevertheless there is an opportunity – AOSD shares many practical problems with BOD – in
particular the issues with testing and verification, and theimplementation of requirements being scat-
tered across several behaviours. Since AOSD is an area of considerable research it is probable that
solutions to some of the AOSD problems will be discovered that are equally applicable to BOD.
71
6.2.1 Spotting opportunities and redundancy earlier
The original set of senses and actions were as follows.
On reflection, after more experience using BOD, the list is too formulaic, reflecting more a sequence
of operations rather than a set of reactive opportunities.
1. The robot needs to find the object
(a) The robot needs to move its hands clear of the table
(b) The robot needs to return to a default pose
(c) The robot needs to see the object
2. The robot needs to pick up the object
(a) The robot needs to ensure its hand is open
(b) The robot needs to move its hand over the table
(c) The robot needs to move its hand to the object
(d) The robot closes its hand around the object
(e) The robot lifts its hand and rotates it so it is holding theobject
Reactive planning benefits from being able to take advantageof opportunities, but the nature of the
task is sequential – there is no point in the robot closing itshand around the object unless the hand is in
the correct place. The list might have been better made with the goals given explicitly:
1. The robot needs to find the object
(a) If its hands are not clear of the table and does not know where the object isthen robot needs
to move them clear
(b) If the robot is not in its default pose and does not know where the object isthen the robot
needs to return to it
(c) If the robot has not located the objectthen it needs to do so
In this way opportunities can be seen, for example it helps flag the question about why the robot
needs to return to a default pose, which turned out to be unnecessary. Of course this was discovered
during the iterative development cycle, but removing this requirement at an earlier stage would have
been better.
As a naive user of BOD at that stage, making me think more aboutprimitive actions as responses to
triggers, rather than just as actions would have been beneficial. Somehow it was too easy to dive into
the list of primitives required and loosing touch withwhy they were needed.
72
6.2.2 Interaction between behaviours
The amount of interaction between behaviours was fairly small, and very well controlled – behaviours
typically just used results from other behaviours (such as the table height). This was largely a con-
sequence of the behaviours each being built around a state since this gives each a reasonably clearly
defined area of responsibility.
This is is in contrast to a DAMN-like system, where interactions between behaviours may be more
substantial which would have had maintainability implications. Adding new functionality could break
the complex interaction and require careful tuning to restore. However some documentation of the
dependencies of one behaviour upon another might be desirable.
6.3 BOD in software engineering
BOD is a type of spiral model software development methodology where the design and development
process proceeds iteratively with the process being repeated as more functionality is added. It is worth-
while considering the different phases of the software development process from a BOD perspective.
6.3.1 Requirements Analysis
BOD does not address the requirements analysis phase, and itwould not be reasonable for it do so.
Working out what something should do should be independent of deciding how it is to be done.
6.3.2 Requirements Definition
The initial step of developing software using BOD is to specify in general terms what tasks the agent
has to perform. This is appropriate for fairly simple tasks but is probably insufficient as a requirements
definition for a complex system, and is a stage following the analysis and specification of requirements.
There can be substantial differences; in the case of pickingup a cube from the table the requirements
might contain a great deal more detail than is needed for BOD.However the point of specifying the tasks
in BOD is so that the developer has a starting point, and the excessive detail added by converting this to
more detailed requirements would destroy this process.
For example, there may be a requirement not to apply more thana certain amount of pressure on the
object being picked up, or the requirements may contain a specification of the dimensions and shapes of
objects to be picked up. They may, also for example, specify performance targets, such as the number
of objects processed per minute, or the maximum acceptable error rate in picking up cubes. These can
all impact the design – the strategy for grasping may be modified if there is a limit on the maximum
pressure that can be applied; but this must be a step before the BOD methodology is applied.
For a complex or rigorously defined system, BOD probably needs an additional step after generating
the likely list of sequences of actions, to confirm that they are not in conflict with any of the require-
ments. Even so, it may prove later that this sequence cannot meet the actions, but it is nevertheless
important not to proceed with the following stages if there is no possibility that all of the requirements
can be met.
73
6.3.3 System Specification
BOD helps system specification by defining the types of operations which need to be carried out and
splitting them out into behaviours. It only considers some areas of functionality, for example it does
not consider the code below the behaviour level. It shares much in common with the Feature Driven
Development process (Palmer & Felsing, 2002), part of Agilesoftware development.
However it discards class diagrams or other graphical devices, and indeed much of the features of
classes apart from the association of methods and attributes. BOD does not exclude the use of this type
of feature but it does not mandate or encourage them.
In Feature Driven Development the class is orthogonal to thefeatures being implemented, whilst in
Feature Driven Design there is a domain model which providesan overall framework into which the
features are built.
BOD has a little to say about the development of behaviours, in the context of the allocation of
tasks. The principle is “if in doubt, favor simplicity” (Bryson, 2001a, page 34), for example primitives
in behaviours are to be preferred to structures in the plan. However most of the guidelines refer to plan
elements such as competences and action patterns rather than behaviours and it does not mandate that,
for example behaviours they should be small and simple, although this is implied by their partition by
state variables.
6.3.4 Software Design
BOD does address the design below the system specification, by the POSH plans and by the state
variables. Because it is an iterative process, the design and the implementation are combined. The
process is neatly summarised as
1. Choose a piece of the specification to work on.
2. Code, test and debug plans and behaviors to build that piece.
3. Revise the specification.
4. Go back to 1.
Keep doing this part over and over until your agent does everything you want it to.
[(Bryson, 2001a, page 34)]
6.3.5 Implementation
BOD uses objects to represent behaviours. This is a convenient way of encapsulating the methods and
state variables associated with them.
The “unit of implementation” is the behaviour, which is a piece of functionality rather than an object
which provides several different services to different components.
In some ways the development of the robot applications felt like Feature Driven Programming, with
the behaviours using functionality from the underlying C++classes.
It was quite hard to limit the number of state variables, but with some effort they were reduced. It
was always worthwhile to re-determine them from the environment rather than use them as a memory
74
of the robot’s previous experiences. For example, the hand position can be determined easily each time
that it is required rather than remembering where it is. Thisallows the robot to be resilient to changes in
the environment, and makes the interaction between the robot and its environment much more reactive,
and consequently more resilient and less brittle.
6.3.6 Unit Testing
Although testable units come out of using BOD there is no recommendations within in about how testing
should occur.
6.3.7 Integration
Integration is to some extent handled by the iterative process of implementation one behaviour and
testing it, and then adding the next behaviour. This is a consequence of the iterative development
feature of BOD and is a very useful characteristic since it avoids delaying integration problems to late
stages in the assembly of the system.
6.3.8 System Testing
BOD does not consider system testing. Regression tests may be developed as part of the iterative
process.
6.3.9 Maintenance
BOD does not explicitly consider maintenance issues directly although part of the justification for the
iterative approach is to make software easy to extend. This was born out by experience in the project
– no great difficulty was encountered in modifying the software to change its behaviour in a fashion
not predicted during design. This is probably a consequenceof the iterative, spiral methodology, since
implementing, testing, refining produces software which has been extensively modified and is thus
capable of further modification provided that too much complexity has not been introduced on the way.
Bryson goes further than this though, by discouraging complexity, both in the general principle that
“when in doubt, favor simplicity” (Bryson, 2001a, page 34) and by advising seeking a compromise in
complexity between behaviours, action plans and actions.
6.3.10 Documentation
Bryson has a considerable amount to say about documentation(for example Bryson, 2001a, page 124-
126) , and the importance of keeping older versions of the software. It encourages self-documenting
code, which is sensible since documentation which is not part of code can quickly become out of date
if not maintained. These recommendations are not specific toBOD, but are practical advice used in
many other types of software engineering. As she points out,however, the reactive plan provides a good
uptodate and concise summary of the operation of the system,something often lacking in an otherwise
well-documented system.
75
6.4 POSH
POSH plans proved very flexible and are easily reworked to change the behaviour of the robot. They
proved resilient, since the failure of the robot to achieve some goal, for example moving the hand to the
cube, would be corrected when the goal was once again addressed.
One perverse problem was that it made it harder to find bugs – during the development the hand
was moved in afor loop until it was close to the cube, and could exit early from the loop if an error
occurred. The POSH plan would call the method again when it found that the hand was not yet near to
the cube.
This resilience made debugging, in this case finding and removing the exit, hard. I spent about an
hour trying to work out why code at the end of the method was only sometimes called.
Because there was initially only one behaviour the class implementing it tended to become cluttered
with methods for controlling the right hand, methods for looking after vision and methods for controlling
the whole robot. The application was probably at the limit ofwhat can be done using a single behaviour.
It seemed natural to do so at the time because the different actions were not distinct from each other as
they might be in a multi-agent application. However it did subsequently prove easy to split the behaviour
up when it became necessary to do so when adding more complexity to the system.
During the early development of the code a few behaviours incorporated loops. A good example
of this was moving the hand to the cube. It turned out that thiswas most convenient to do during
development because it allowed experiments with varying step sizes and moving the hand to the object
in the X axis before considering the Y axis to be done easily. Once the code was working the loops were
unrolled and the POSH plan used instead to drive them. This meant that error handing in particular
changed – rather than the loop detecting an error such as an inability to move the hand to the cube
because it was out of reach, the POSH plan had handling a missing hand position as a higher level
goal. The unrolled loop now contained no error handling, since error conditions were detected as higher
priority goals, with a corresponding simplification and theopportunity for the robot to react to higher
priority events.
6.5 Use of classes
In BOD, a behaviour is the collection of methods needed to accomplish some task, and which are linked
together by sharing the same attributes. The behaviours therefore represent some sort of activity, such
as Grasping (the cube), or in the case of the work done simulating macaques discussed in Section 2.6.2,
activities such as Grooming, Exploring and Feeding. In a conventional object oriented paradigm the
classes normally represent nouns. For example there is an iCub object which is composed of sub-
systems, which are themselves arms and legs. This allows inheritance and polymorphism to be used.
One area in this work where there is a need for polymorphism, and tying more closely a POSH
competence to its behaviours in the work described here is ingrasping. The code was originally written
to just use the right hand, but later extended to use the left hand. However the POSH competence and
behaviour only handled grasping with the right hand. The operation could have been identical for both
hands in other circumstances, and some solution using a state variable might have worked.
76
6.5.1 Encouraging component reuse
Behaviours are defined by their states, which makes them easyto partition for initial implementation
but does little to help with code reuse or extending the implementation. In its present form BOD does
not explicitly encourage the developer to implement simple, general purpose behaviours which can
be specialised through polymorphism or inheritance. Nevertheless behaviour reuse is relatively easy
because of their implementation in classes.
The grasping behaviour of the right hand in the initial task could have been re-used as grasping for
the left hand in the extended task, or perhaps grasping different shaped objects. Picking up a flat sheet
or a cup by its handle is in many ways similar to grasping a cubeand perhaps much of the original
behaviour could be reused. During the extension work reuse of the left hand grasping code was tried
and it proved easy.
However reuse of the plan was not possible, except by “cut-and-paste”. This is a consequence of the
jyPOSH plan syntax which does not allow the same competence to be used with a different behaviour,
and not a criticism of POSH in general.
It could be implemented in jyPOSH, in a rather clumsy way, by having a state in the behaviour which
said which sort of specific behaviour it was. If this has to be done, it would be better to implement this
through inheritance, which means that the POSH plan being executed would require a reference to the
particular object it was using.
A more general implementation of POSH would allow the behaviour which it is using to be ex-
pressed as a deictic variable. In discussions with Joanna Bryson, it appears that the Nomad 200 robot
had this capability, since the 16 different sonar sensors were each represented by an object and the same
competences was used for all of them. However this feature has been lost in the more recent jyPOSH
implementation.
Reusing part of the plan through inheritance is not possible, but is probably not desirable in the
general case. A slight modification in a competence, perhapsinserting an extra trigger, is likely to
have such a dramatic effect upon its action that it may be better not to to try to do this. In any case
competences are short and highly specialised so that they often bear little relationship to each other.
However if two behaviours are related through inheritance then it is likely that they will use the
same plan, but one really may need an extra trigger. In this case the competence could be written for
the more complex case and the superclass behaviour include atrigger which always returns false so the
action is never released unless the sense behind the triggeris replaced.
6.6 Non-interruptible tasks
It is possible to switch behaviours in POSH. This normally happens when a higher priority goal is
triggered, in the robot simulation the failure to pick up theobject within the timeout interval will trigger
a high level goal which will remove the object, reset the robot and request a new object in a new location
on the table.
A lower priority goal can be satisfied in scheduled POSH if a higher level goal which is being
satisfied has a lower frequency of inspection that the time taken for it to carry out a step. The idle time
77
can be used to execute lower priority goals.
However it is not always desirable to rapidly switch from onegoal to another, since it may not be
desirable to interrupt the current behaviour.
For example in grasping the fingers are slowly closing and this motion needs to be stopped as part
of the grasping behaviour. One way of avoiding this is to makethe action non-interruptible by making
it a single action, but this loses the ability of POSH to switch to higher priority goals and prevents
competences from being used to implement the different steps in this action.
Several solutions are possible:
6.6.1 A cleanup action
Each behaviour could have an action which is always called before the goal is changed away from it.
This action would ensure that the robot is in a stable state. POSH would send a signal to trigger an action
in the currently active behaviour when it changes goals to signal it to enter a “safe” state. A problem
with this is that the cleanup itself might be rather complex and benefit from using a competence to carry
out its steps. This can be accommodated if the behaviour has away of responding to the POSH engine
to relinquish control and allow another goal to be pursued when it has finished.
Even this has problems though, for example the urgency of thehigher priority goal becomes an
issue. If the robot is at risk of falling over, then a goal to keep it upright needs to become active very
quickly regardless of its handling of cubes. It is far betterfor the robot to stay upright and drop the cube
than to fall over and be damaged.
In a very complicated machine there is a hierarchy of urgencyof behaviours based upon likely
consequences. This might be implemented as a timeout; that the interrupting goal indicates to the
POSH engine how long it can wait for cleanup to finish. This needs to be passed to the interrupted
behaviour so that it can make decisions about what it can do inthe alloted time.
6.6.2 Current goal allowed to continue
Scheduled POSH would allow the current goal to continue whenthe higher priority goal was triggered,
using idle time not needed by the higher priority goal. This would work if the two goals were not in
conflict, for example the robot can continue to pick up the cube whilst attempting to stand upright. The
problem is that in many cases the two goalswill be in conflict – perhaps the robot needs to use its arm
to stop it falling over. The question of simultaneously active goals is considered further below.
6.7 Simultaneously active behaviours
Proetzsch et al. (2007) criticised the use of BOD as a robot architecture, as the strictly prioritised plans
that BOD uses are appropriate for achieving the high level goals but are deficient in low level motion
control.
For example they do not permit more than one behaviour to run at the same time. This was not a
problem in my work, since the robot was only undertaking a single task at a time, but would clearly
78
have been an issue if the robot was, for example walking, at the same time as passing objects from one
hand to another.
The root of the problem is the strictly prioritised goals. Itis easy to envisage applications where
several goals can be achieved at once, for example controlling its motion and communicating with a
human at the same time. This is a consequence of eligible goals being run solely on the basis of priority.
When the robot was executing the competence to close the fingers around the cube on the table I was
often asked if the task had hung since there was no apparent movement because the finger movement
was not in view.
6.7.1 Interaction between behaviours
Criticism that behaviours cannot interact with each other though is unfounded. Behaviours can interact
with each other and it was essential that they did so, and theytypically do so by exchanging state
information.
For example the behaviour concerned with finding the table height interacted with the other be-
haviours to provide them with the table height, and the behaviour controlling the processing of the
yellow cubes; largely done using the left hand, communicated with the behaviour for picking up the
cube from the table in passing them from one hand to the other.However the communication was never
at the level of seriously influencing behaviour or arriving at a compromise between two behaviours.
A problem which could have caused serious repercussions waspath planning. For the task it was
not necessary to implement path planning but the interaction between the behaviours controlling the left
hand and the right hand might have been hugely complicated bytheir potential physical interaction. If
the left hand is to be moved to a particular position to receive the yellow cube from the right hand, and
is unable to do so because the right hand is in the way; then theright hand would have to be moved. But
if the right hand is in its present position because of some higher priority goal then that goal would no
longer be satisfied and the hand would be moved back again as soon as the POSH engine re-examined
the plan, possibly moving the left hand out of the way.
6.7.2 Split of functionality between behaviours
Behaviours are somewhat arbitrary in their definition. Theyrevolve around the state variables that they
contain, and so there is no one-to-one correspondence between drives and behaviours. This is partly
because achieving a particular goal may require examining or changing several different state elements
and thus involve several behaviours. But there is some confusion between the state and the behaviour
– in particular the behaviour is “what the robot is doing” butis actually “what piece of information the
robot has”. For example the behaviour calledRemovingYellowCube has a piece of state information
which says what it is doing:
1. The robot is trying to hold the cube
2. The robot is holding the cube
3. The robot has got rid of the cube
79
This is actually implemented as three boolean variables, but they essentially hold one piece of in-
formation between them. This worked very well. If some otherway of processing the cube was found,
perhaps by dissolving it by pouring water on it, then other variables might be needed which could
fragment the behaviour but it does not seem likely that this would be a serious problem.
One very minor criticism of the .lap syntax as used in jyPOSH is that it would be nice to make
it clear which behaviour a particular action belonged to, because in a large system much time could
be spent searching different behaviours for the particularaction when trying to understand the code.
However this could be fixed without code changes in the actionname.
6.7.3 Relative priority ordering cannot be changed
One criticism of BOD I have heard several times is that the order of the priority of goals cannot be
changed. In part this is a consequence of the BOD methodologysince an early step is to decide the
order of priority. It would be possible to allow priorities to be altered, but the inability to do so did not
affect the tasks undertaken for this project.
Hypothetical situations can be devised for changing task priorities. An example might be recharging
batteries. A mobile robot may increase the priority of recharging as the remaining energy in the battery
declines. So as the battery runs down the priority floats upwards until it becomes more urgent than
the task that the robot is performing. An alternative is thatthe robot calculates a cost to charging the
battery if the charging points are far apart. If it happens tobe close to a charging point then it may be
worthwhile for it to partially recharge its batteries even if they are only partially discharged.
Different implementation can be considered, for example having a priority level as a variable or
having the ability to give a priority boost to some goal. A problem with these schemes is that the system
for changing priorities has to understand more than just thegoal whose priority is being changed, but
the reasons for the priorities of the goals which are effectively being downgraded.
There are problems with changing goal priorities, in particular the the complexity of managing dy-
namic priorities (as Bryson comments on page 70 of Bryson, 2001a, although here she is concerned
with lowering priorities) which makes alternative solutions more attractive. Priority shifts can be im-
plementing using POSH in two ways currently:
• Push the decision making into the trigger. Charging the battery is given a high priority but the
trigger for it looks at the current charge level and the proximity of a charging station and makes
the decision about whether to charge the batteries on this basis.
• As two drives. The higher priority one is released if energy level falls below a critical level, whilst
a lower priority one is present so that energy can be obtainedif the robot is not doing anything
more useful.
However I do not believe that varying short term goal priority is generally needed in any complex
way that cannot be handled by the above methods. A quite different question islong term goal priority,
or whether the robot needs to radically “switch modes” because of a step change in environment. For
example a lander travelling to the Moon will operate in one mode whilst in the payload bay of the
vehicle carrying it, perhaps just using its camera to make observations of the distant moon, but switch to
80
another mode, acting as an autonomous mobile robot, when it has landed. It is quite likely that some of
the behaviours and the code below them will be the same but that it has two POSH plans and switches
between them when its environment changes. However this does not require any modification to POSH,
but support for it would be required in the implementation ofthe POSH engine.
6.8 Multiple goals
POSH plans do not seek multiple goals simultaneously, except that a lower priority goal may be pursued
when the rate of checking is throttled back for a high priority goal. The original Nomad 200 work which
led to the development of Edmund and thence BOD had six goals,which could be split into ones which
could be achieved very quickly or ones which took an appreciable time. The goals were:
• Return to base if battery energy falls below threshold
• Set speed proportional to distance to nearest obstacle
• If robot finds itself in an area not previously mapped then mapit
• Remain near base if not busy
• Detect approaching people
• If a human waves their arms then follow the human until they stop moving
It need only be carrying out one of these goals at a time. However a more sophisticated robot such
as the iCub has subsystems which can be used independently ofeach other.
A critical difference between the Nomad 200 and an iCub-likerobot, is that the latter has the capacity
to achieve more than one goal at the same time. It could, in principal at least, simultaneously achieve
the goal of getting to a new location using its legs and cameras whilst at the same time perhaps carrying
out some other task with its hands, such as sorting a pack of cards. Neither of these tasks will take
negligible time, as perhaps setting the speed of the Nomad 200 did, so it would be carrying them both
out simultaneously.
It would need to share the camera system between the two tasks, but it could certainly carry out
these goals simultaneously. It would of course be possible to designate one task as higher priority, for
example sorting the pack, and then carry it sorted to the new location; but this would take longer than
in a similar robot which had software capable of carrying outboth tasks at the same time.
Whilst this is perhaps unrealistic for a laboratory robot, itwould certainly be the case for a highly
sophisticated machine which might implement action section using BOD. For example, probably the
most functional “robots” in existence, airliners, have a flight management computer which is respon-
sible for following a pre-determined flight plan, which requires the simultaneous goals of flying the
aircraft at a particular altitude, heading and speed, and atthe same time minimising fuel consumption
by setting engine parameters and pumping fuel around the aircraft to maintain the aircraft’s trim as it is
consumed. In this case there is no real priority, except perhaps maintaining altitude is more important
than minimising fuel consumption.
81
6.8.1 Non-conflicting goals
The key to simultaneously achieving independent multiple non-conflicting goals is resource availability.
The reason that a humanoid robot can achieve the twin goals ofchanging location and shuffling a pack
of cards at the same time is that it has the resources necessary to do so. It can achieve the goal of
changing position because it has legs, and it can perform some action with its arms at the same time
because these are otherwise unused resources. Even so, it would be necessary to share some resources
between the tasks, for example the cameras.
It would be possible to have two independent POSH plans of course, one driving the legs and the
other the arms, but this would not permit changing to anothertask which perhaps used the legs and an
arm, or two independent goals each requiring an arm each. It would also cause problems with resources
which needed to be shared, for example the cameras or the torso.
This is a key difference between the use of POSH in a sophisticated robot and in other applications
that is has been used for. In these, including the Nomad 200 robot, it is used to control one or more
agents, but each of which is only performing one substantialactivity at a time. It might have a higher
priority goal which it can rapidly achieve when it needs to attend to but it can return to its single
substantive goal when that is complete.
An important difference between BOD and other schemes such as Subsumption or IB2C is that
BOD only has one behaviour active at a time. Behaviours in a single agent can interrupt each other
and use results or even actions from other behaviours, and lower priority goals can “interrupt” higher
priority ones if a maximum rate is specified for the highest priority one but they never run concurrently.
Concurrency in BOD tends to be when several agents are running in parallel – for example in the
macaques model there are many agents all running one of two plans in their own thread. So several
agents might be simulating monkeys but at any instance may berunning different competences. They
can of course interact – and it is essential for the agents to do so to simulate social interaction.
It is possible to control a robot in this way, as shown by the Nomad 200 work, with different agents
controlling different limbs, but the interaction between agents seems problematic. There is an overall
goal for the system and if the agents are jointly responsiblefor it then they will have to interact in a
subsumption type way. But it is difficult to design such a system if it is highly complex.
For picking up cubes, there might be an agent controlling each group of joints (shoulder, wrist) for
example, but this system would lack the overall supervisorycontrol and could only achieve its goals
through emergent behaviour. Emergent behaviour is complex, somewhat unpredictable and prone to
big changes when small adjustments are made, and likely to react unpredictably to the failure of a
component, so developing such a system may be very difficult.There is the problem that the right hand
might literally not know what the left hand was doing, or at least why it was doing it.
A system which had many goals, and for which achieving several simultaneously, would probably
have many more than the 5-7 entries in its Drive Collection that Bryson recommends (Bryson, 2001a,
page 62). However this is really a consequence of the requirements for the robot rather than something
which BOD as a methodology can specify. If the robot needs to perform dozens of different goals then
there will inevitably be dozens of different entries in the drive collection unless they can somehow be
combined. This is inevitable if BOD is to be used for very large and complex systems.
82
Would it be possible to have several POSH plans running in different processes in the robot? It
would, but only if one plan never needed to use the resources that another does. For example one POSH
plan could be looking after understanding natural languageand another navigation, but they would
either need to be combined, or very careful through given to the design, if the robot needed to ask for
directions.
6.8.2 Conflicting goals
BOD is good at resolving conflicting goals, and in the currentversion Strict POSH has an implicit
assumption that all goals conflict. Scheduled POSH allows multiple goals to be pursued simultaneously
but places the onus upon ensuring that they do not conflict explicitly on the developer, who must prevent
this using the trigger conditions. Goals conflict when they need to use the same resources or when they
are working in opposite directions.
A physical manifestation of this might be that the goal of walking to the North Pole is in conflict with
walking to the South Pole and attempting to achieve both simultaneously would result in no progress.
Strict POSH would do the highest priority of these, whilst Scheduled POSH would allow the lower
priority one to interfere.
6.9 Abstraction
The robot programming tended to work downwards from the POSHplan into the fine detail of, for
example, grasping. It would be better if the behaviour was a set of methods specifying the action
needed to be done, but delegated doing that action to some other classes; and so it acts like an interface
rather than a class. The reason for this is that different aspects of the behaviours need to interact with
different objects in the system. In this way the behaviour would look like a Facade object oriented
model rather than implementing all of the functionality itself.
6.10 Scaling
During the work I was acutely aware of the problems I would face if I tried to scale the task. Some
robots perform very simple tasks, for example the iRobot Roomba vacuum cleaning or floor washing
robots, and BOD could be used very easily to control them. However many robotic machines have
extremely complicated behaviour, an example mentioned previously is the flight management software
in an aircraft. Many pieces of equipment not usually thoughtof as robots could benefit from reactive
planning. The most complex software is that several people are involved in its development and no
single individual even understands how it all works. Consequently it must be modularised into pieces
which have clearly defined interfaces and do not have unexpected interactions between them.
A particular problem was that it was easy to get sucked into low level detail when developing a high
level plan, because some low level detail appeared in the .lap file and modularisation would help with
this too.
83
A drive collection for such a system would need to handle manyconcurrent, but non-conflicting,
goals simultaneously.
In some cases it would be possible to split out a component as afunctional unit and implement it as a
standalone device containing its own set of goals and POSH plan, or several identical devices following
the same plan as separate agents. However even where this wasdone it is possible that some devices
would be needed as non-shared resources by the whole system.
BOD could probably be used for such a large system if the drivecollection were considered sepa-
rately from the behaviours, and the behaviours were kept independent of each other as far as possible.
It would require some modularised compilation of the plan sothat different people could work on dif-
ference competences, and most importantly the capability to identify, and pursue non-conflicting goals
simultaneously.
6.11 State variables
The behaviour and the plan are the two things that define a BOD implementation. However the senses
tend to be different from behaviours inasmuch as they are often, or should be, stateless.
Some senses are used by several behaviours, and it was not clear where they should be. The method-
ology makes their position plain, since behaviours are built around state variables, but some states are
so simple that encapsulating them in their own behaviour makes a proliferation of tiny behaviours. An
example is in the task extension given to the “naive user”, which requires a new state variable which
records whether the cube has been held or not. There is no action associated with this state since its
only effect is on the trigger conditions in the plan.
It would have been possible to split them out and put them intotheir own “behaviour” but to avoid
the proliferation of degenerate behaviours or the construction of an “odds and ends” behaviour fairly
arbitrary decisions about where to put them were made based on where they were used most.
6.12 Multiprocessor systems
The iCub robot contains several different microprocessorsbut they are effectively hidden from the
developer of the high level functionality by the PC104 coordination processor and the YARP software.
However if this were not the case, how would BOD be used in sucha system. In an agent simulation it
would be easy to “farm” different agents out to different processors. But if the system was being used
to operate a robot, and multiple processors were needed to provide the necessary computing power or
provide hardware access to different components how would BOD be used? Because it is essentially
single threaded inasmuch as only one goal is being pursued atany instance there is the risk that all but
one of the processors stand idle. Perhaps several goals could be pursued simultaneously where they do
not interact.
84
6.12.1 POSH plans
It is clear that POSH in its current form cannot handle this case. However it may be a question of “horses
for courses” – POSH is an arbitration scheme for action selection and perfectly suitable for the types of
problem which require this solution. However the thesis is about using it in robotics, which may require
several behaviours active at once so thought needs to be given to extending BOD to handle such cases.
Nevertheless the experiments with grasping demonstrated that BOD was perfectly suitable for or-
chestrating the movement of individual fingers, despite my own initial skepticism that this would be the
case.
6.13 Testing
For a methodology to be commercially or socially useful, it needs to be amenable to testing and verifi-
cation. This is a problem with most fusion behaviours – if thefinal behaviour cannot be easily predicted
then how can it be validated? The manufacturer has to be confident that a device incorporating it will
not malfunction in normal use, and will not dangerously malfunction if exposed to conditions outside
of normal use.
The timeout on each optional competence or action pattern may even be a little pernicious since
it increases the complexity of the testing. The ideal would be that each behaviour in a competence or
action pattern is made “safe” so that no harm will be done if the competence or action plan is interrupted
at this point. However this would be difficult to achieve in many cases–what is really needed is some
mechanism for the behaviour to enter a safe state before a different goal is pursued.
Arbitration methodologies such as BOD are more predictablethan command fusion types and so
may be much easier to test and verify. A methodology where multiple behaviours interact may give
rise to unpredictable and undesirable activities which require the system to be submitted to much more
careful analysis to detect.
An advantage of reactive action selection is that it has relatively little internal state information in
contrast to sophisticated models built up using deliberative planning – for example there may be no
lengthy history affecting current behaviour, although this changes when learning is introduced.
Learning in the field should not greatly modify the behaviour, but control when different behaviours
are performed and perhaps make small changes in parameters within the behaviours. Otherwise there is
a real risk that any testing of the software will be rendered meaningless.
However all of this is moot if arbitration methodologies arenot sufficiently rich to allow the desired
functionality to be implemented.
Four types of testing are possible at present:
• Static examination of the POSH plan
• Reducing the plan so that only one goal is met to run limited unit tests
• Running unit tests on the classes underneath the behaviours
• Running in a realistic environment for an extended period
85
In general plans are easy to test because the BOD guidelines propose only having a few elements
in each competence or action pattern, which is consistent with it being held in the short term memory.
George Miller’s classic discovery (Miller, 1956) that a maximum of only 9 items can be held in short
term memory at a time probably drives this as it does much elsein software engineering. However
as plans become more complex it is not obvious what the test plan for one should look like although
an automatic way of generating the states needed to trigger different goals automatically as part of
regression testing would be desirable.
Testing reactive plans is an important question, since if they are to be used to their full potential in
commercial products then this question needs to be properlyaddressed. A reactive plan controlling a
simulated robot removing cubes from a table, or simulating the behaviour of groups of animals does not
require exhaustive testing because the cost of failure is not very great.
What seems to be lacking is a way of identifying a rare combination of circumstances which will
cause problems, perhaps through interaction of two behaviours especially in scheduled POSH. This is
not a problem in simple plans but the number of combinations may cause difficulties in complex plans.
One possibility is that a tool like ABODE can simulate the operation of plans and perhaps indicate
possible conflicts if it understands the resources which each action uses.
Some work in formal methods for reactive plan verification has been done (for example Kapellos
et al., 1995; Koo et al., 1999) but it needs to be applied to POSH. A difficulty might be the lack of
a clear link between requirements specification and the POSHplan. Given the logical structure of a
POSH plan it is likely that a tool could be produced which would carry out some mapping between
formal requirements and the POSH Basic Reactive Plan automatically.
6.14 BRPs as part of something bigger
An area which some thought has been given to by the author is whether a BOD sequence is a sub-set
of a larger and richer control structure. There are three types of structure in a Basic Reactive Plan, the
Drive Collection, the Action Pattern and the Competence. The Action Pattern is a sequence of actions
which are executed in sequence until they have all completedor the list is terminated by one of them
returning fail. In principle POSH allows APs to be specified with elements that can be executed in any
order or in parallel, although Bryson comments that this is not particularly useful (Bryson, 2001a, page
64). I cannot see a use for them either, since actions executed in parallel can be implemented inside an
action and out of order execution probably normally means that the actions are not dependent upon each
other and so could be executed in parallel.
However Bryson says that Action Patterns should only be usedfor sequences taking a fraction of
a second (Bryson, 2001a, page 69) because they prevent the agent having the opportunity to do other
things. This advice has weakened since Bryson’s PhD as an examination of applications written since
then, such as the macaque simulation and the Unreal Tournament agent, show that is not rigorously
followed, although generally the last element of the actionpattern is a persistent one and the first few
are not, for example (from poshbot):
(AP avo id ( m inu tes 10) ( s top−bot r o t a t e then−walk ) )
86
Bryson clearly discriminates them fromchained competenceswhere the following action in a se-
quence was triggered by the previous one. I converted several chained competences into action pattern
because I found a proliferation of variables hidden in behaviours. For example the longest action pattern
was the sequence of operations to move the cube from the righthand to the left hand:
(AP move−cube−to−lhand
( a l o s e c u b e a m o v e l h a n d i n a c u p l h a n d
a r e l e a s e c u b e a o p e n r h a n d a m o v e r h a n d c l e a r a g r a s p l h a n d ) )
The problem with using a chained competence here is that the robot is unlikely to have a higher
priority goal and having a state whose sole purpose was to sequence events dependent from each other
added complexity without gain.
One problem I did have was that this pattern was inside a competence triggered by the robot hold-
ing the cube, which in turn was triggered by a competence where the cube is yellow. Between the
a openrhand and agrasplhand the cube is not being grasped by any hand (it falls from the right hand
to the left hand) and so the competence is no longer triggered. The meaning of “holding the cube” was
blurred to fix this problem. Although this worked I was troubled by the subtle interaction between a key
state element for the upper layers of the plan and a lower layer.
A solution is to modify the BOD methodology to recommend thatthe state elements are more clearly
defined in the early steps. These definitions can be changed iteratively but I was fortunate the meaning
of “holding the cube” could be changed without dramaticallyimpacting other areas of the plan. This
is not a problem in the simple implementation but could causesevere difficulties in a larger and more
complex plan.
There is more work to be done in looking at chained competences, perhaps introducing a more
explicit way of showing that the competences are chained together in the plan.
It is meaningless to have a competence as an element of an action pattern but there might be circum-
stances where one of a set of actions needs to be performed, orperhaps an action optionally performed.
For example the areleasecube action is releasing the “virtual magnet” in the simulator, which only
works if the Simfork simulator is being used. If the normal simulator is being used then it does nothing,
but there might be circumstances when this action should notbe performed and hiding what might be a
key check inside the action makes the action pattern less clear.
6.15 Test and verification
Unit tests of the underlying C++ classes were done and testing of the behaviours as they were built up,
but it was difficult to test the plan and especially to ensure full code coverage of the behaviours.
One problem rarely considered in academic literature on reactive planning is strategies for testing
the action selection plan. Indeed most papers on artificial intelligence or software generally describe the
work undertaken or the scheme but make no mention of testing or verification. There is a fundamental
difference between computer science research and industryand is an issue in transferring techniques
from the laboratory – a technique which gives rise to code which cannot be tested and validated properly
is practically useless in the “real” world since demonstrable reliability is needed for it to be incorporated
into a product.
87
Testing reactive plans is then a vital question, since if they are to be used to their full potential in
commercial products then this question needs to be properlyaddressed. In circumstances where the cost
of failure of a reactive plan may be very great, it needs to be fully tested and possibly even validated
with a proof-of-correctness, and consequently such techniques must be available.
A simple POSH plan (or any other type of reactive plan) is easyto understand but as it gets more
complicated then it becomes harder to understand and to test. To some extent it is modular, but there
are potential interconnections where none are expected. This is particularly true of Scheduled POSH.
In particular understanding whether software is behaving correctly is linked to understandingwhat
it is meant to do – linking the requirements into the tests. Anarea in which BOD works well is in putting
the requirements into the design, inasmuch as the first step involves specifying in general terms what
the robot is to do.
The first stage of using the BOD methodology is to specify in general terms what the agent is to do,
probably in a more detailed specification of likely senses and actions, with some linkage at this stage
back to the requirements.
One solution to testing might be to build into the engine a wayof triggering each drive in turn, or
of triggering each element of a competence for test purposes. However this will not work if a trigger
requires a particular set of actions to have been performed previously, for example to put the behaviour
into a particular state. Scheduled POSH may be particularlydifficult to test in this respect since if more
than one drive element is active there may be rise to subtle interactions between behaviours.
If the interactions between behaviours is limited and the system has little memory then it might be
possible to unit test each action in each behaviour, although there will have to be a suitable ordering.
In a complex plan it is clear that not all paths through the code can be covered, which is also true of
most programs, but it is not even clear which are the most likely paths to be traversed.
6.16 Expanding to a large system
All of the issues discussed above become more significant if POSH were used to develop a very large
system, for example if the robot had dozens of different activities in its repertoire instead of just handling
cubes.
6.17 How BOD should be extended
6.17.1 Extensions to the BOD methodology
It would be helpful to encourage more thought and a clearer definition of state elements in the early
steps. Whilst state elements can be changed in a later iterative cycle, in a large system this can have
major knock-on effects.
The boundaries of the methodology should be more clearly defined. For example the implementation
of behaviours discussed in Section 6.1 would benefit from clearly specifying that Behaviours arealways
implemented as objects in BOD, and that implementing them inother ways is outside of the limits of
88
BOD; but that it is perfectly acceptable to venture outside of the limits of BOD if the application requires
it.
Action Patterns are a useful sequence structure and widely used. However there is an overlap be-
tween these and chained competences. Extending the methodology to handle chained competences in
an explicit way – making the state that they are using clearer, especially in the cases where there is
only one state would make the resulting code easier to comprehend. Note that in these cases a chained
competence is really a state machine.
6.17.2 Extensions to POSH
This work has brought to light an area for improving POSH – that an agent can be too single-minded
and will only pursue a single goal even when it is capable of pursing more than one goal. Scheduled
POSH allows POSH to pursue more than one goal, but doesn’t consider how the pursuit of different
goals might conflict or work together.
The key to several goals operating at the same time is deciding whatresourcesare needed by the
actions needed to achieve different goals. In this way several goals can be sought simultaneously without
interfering with each other. For example the robot could in principal perform some task with its hands
at the same time as walking to a new location because the two activities use different limbs. There
would be some overlap with the vision since they would be required both for looking at its hands and
for navigation. Schemes for sharing the camera can be devised, perhaps making it a shared resource
which is available to any behaviour in limited time slices (i.e. rotate to desired direction and take image,
then make available to other behaviours).
The modified scheme for the drive collection would be
• Look for highest priority goal not running but which has beentriggered
• Get a list of the resources it needs
• Start a process running the sequences needed to achieve thatgoal
• Repeat until the overall drive collection goal has been achieved
When a goal has been achieved, the process pursuing it exits and all resources allocated are returned
to the pool of available resources.
This simple scheme has two disadvantages. The first is that some running competences for other
goals will be using resources which the highest priority goal needs. They will need to be aborted.
The second disadvantage is the possibility of deadlocks, for example if a behaviour requests an
unanticipated resource. There are numerous solutions to the deadlock problem but the simplest is to
require all resources that an activity needs to be requestedbefore it starts.
The new scheme is now:
• Look for highest priority goal not running but which has beentriggered
• Get a list of the resources it needs
89
• If those resources are only being used by lower priority goals then stop them
• Allocate the resources needed to the highest priority goal
• Start a process running the sequences needed to achieve thatgoal. The process would be running
its own POSH-like engine but with the competence replacing the drive collection as the structure
at the bottom of the slip-stack
• Repeat until the overall drive collection goal has been achieved
6.17.2.1 Resources
A resource may be a physical resource such as the left arm or the right leg, or it may be a concept, or
an attribute attached to other resources, such as “water/electricity” to prevent a hypothetical household
robot from handling electrical devices at the same time as water.
It is possible to incorporate resources into the trigger forgoals∗ but this would cause problems with
maintenance since someone modifying a behaviour would haveto be aware of all of the goals that it
was used in and modify the triggers.
6.17.2.2 Cleanup
The second change suggested by this scheme is a way of cleanlyaborting a drive element. POSH
switches to the highest priority goal leaving the previous activity in whatever arbitrary half-completed
state it happens to be in at the end of the most recently executed action. This can be very undesirable
since it is not possible to just jump out of a complex activity. An example scenario would be drinking
a cup of coffee when the telephone rings. The cup has to be put down safely before any higher priority
goal can be addressed.
It would be possible to have a very high priority goal of “if drinking coffee and activity interrupted”,
then put the coffee down but it is better to keep the goal and the steps needed to terminate it together, so
this needs to be incorporated into the drive.
There should be two levels of “stop” available for any goal, one which is high priority and the
second which is normal priority, although they may in practice execute the same code. One level is too
few since the activity may take an appreciable time to finish normally but the action may need to be
stopped when some damage, for example making the task much slower to restart, are acceptable.
The “stop” drive element is very similar to the drive elementfor carrying out the action, a reactive
competence with a set of triggers for actions within it, so would be represented by a POSH sequence.
It would be executed asynchronously however, since the action being performed by the goal it was
interrupting may be a lengthy one unless interrupted by a signal from the newly running “stop” drive
element.
Thus to Bryson’s original three elements in a drive collection entry (Bryson, 2001a, page 62) would
be added competences for a normal stop and an emergency stop,and each behaviour (or possibly each
action) would have added to it a mechanism for retrieving theresources that it uses.
∗My thanks to Ben Binsted for suggesting this
90
This approach would have the benefit of allowing POSH to make better use of multiprocessor sys-
tems, since different goals can be pursued on different processors.
It would also allow actions to be written as blocking methodswhich take an appreciable amount of
time, although this can be done already if Scheduled POSH is used but some activities really are better
written as lengthy loops. Several times I felt hamstrung by the inability to take an appreciable amount
of time over an operation. The ability to cancel an ongoing drive element and replace with a different
one would keep the quick reactive responses that is one of thebenefits of POSH.
6.17.2.3 Impact on BOD
Adding resources to POSH will undoubtedly impact BOD because these will have to be considered as
part of the design. At the time that the actions are loosely defined then the resources needed for those
actions will be considered. As with states, the precise meaning of each resource must be specified –
although modifying that specification later would change which goals could be achieved simultaneously
but should not affect greatly the behaviours themselves.
6.18 Non-BOD related issues
Some issues discovered during the work have little bearing on BOD but are nevertheless worth com-
menting upon.
6.18.1 Programming languages
A mixture of Python and C++ was used for the work, with the POSHplan itself is written in a syntax
reminiscent of Lisp.
This worked well, the disciplined approach embodied in the C++ meant that the lower level classes
were efficient and conceptually easy to understand, whilst the Python allowed rapid prototyping. Even
so, I still spent a great deal of time worrying about the minutiae of precise kinematic positions.
C++ is a very expressive and powerful object-oriented language. It has disadvantages, in particular
is rather unforgiving of errors and requires considerationof irrelevant to the problem being solved, such
as memory leaks and dangling pointers, and lacks built-in support for multi-threading. Many of these
deficiencies are corrected by Java, which also adds a very extensive runtime library. However C++
remains a popular choice amongst robotics researchers, in part because of perceived poor performance
of Java and a desire to use a common language throughout the robot platform, including embedded its
microcontrollers.
However the use of Python for higher level control resulted in faster development and meant that
there was a clear abstraction layer between the detailed mechanics of controlling the robot and achieving
higher level, more complex, goals. Nevertheless even this had problems, particularly in precise control
the robot’s movements. I spent a great deal of time experimenting with minute changes in robot position,
whereas it would have been preferable to have been able to specify where I wanted the arm to move
to in terms of my goals, and not in terms of position. For example the developer might want the left
hand to move to “where the right hand can drop the cube into it and don’t hit the robot or the table on
91
the way”. It would not be hard to modify the C++ classes to add path planning and concepts about the
relative position of objects and the robot’s limbs. Howeverthis would still be hard to express in Python
in an intuitive way, far less C++.
It could be nevertheless expressed in more or less any language, but if the purpose of a programming
language is to abstract the problem from the details of implementation then neither C++ nor Python
would be doing a very good job. At some point the request wouldhave to be converted into a set of
specific instructions based on the position of the robot and the table but it should not be the concern
of the person developing the higher level code to worry aboutthese details; for them it is a question of
representation.
There is a strong case for using a higher level language tailored to robotic programming. (Wahl
& Thomas, 2002) describes long-term goal of these technologies, in an industrial robot environment,
to allow the robot to make decisions about the details and order of assembly, including consideration
of the forces which need to be applied in doing so and path planning to avoid obstacles. These lan-
guages remove the requirement for the programmer to micro-manage the dynamics and kinematics of
the task and focus on what needs to be achieved rather than theprecise movements needed to achieve it.
Whilst research robots typically use middleware such as YARPand URBI, these are intended to provide
abstractions of devices and networks rather than abstractions of higher level functionality.
6.19 Grasping
One perverse advantage of the grasp being highly unreliableis that for experimentation the magnet grab
is usually disabled. This means that the robot drops the cubeor moves it around on the table allowing
various error states, such as not being able to see the cube, or not being able to reach it, or the hand
moving out of view when reaching for the cube, to be tested. Most of the work with evaluating POSH
has been done with the grab functionality disabled simply because it causes the robot to operate outside
of the original limits of the program. It allows a view to be formed of how resilient a POSH plan is
in the face of unexpected behaviour and how easy it is to modify it where deficiencies are found in the
plan.
The iCub simulator usually manages to grasp the cube but drops it as it attempts to lift it from the
table. It then attempts to grasp the cube again, this time perhaps pushing it to the edge of the visual
field. It is examining, and modifying, the POSH plan to cope with errors which has proved particularly
fruitful in understanding its limitations. This is a realistic exercise since it is frequently the case that
existing functionality needs to be extended to cope with changed circumstances.
The problems with grasping represent a limitation of the simulator, and are not any indication of the
usefulness or otherwise of BOD. The only impact that BOD had was in the choice of control mechanism
for closing the fingers.
Grasping was at one stage implemented asfor loop since it was seen as an atomic action which
would not be interrupted, and I was particularly keen to shutoff the drive to the iCub finger as soon as
it touched something, so didn’t want to rely upon a large bodyof code such as the POSH scheduler to
ensure this. However, once it was understood that the physical iCub robot cannot apply sufficient force
with its fingers to damage them, it was converted back to a POSHcompetence so that the robot could
92
be interrupted to perform higher priority actions if required.
93
Chapter 7
Evaluation of the project
7.1 What has been achieved
The project has achieved an evaluation of the applicabilityof BOD to complex robotic systems, using
the following activities:
• A task for a simulated complex robot was developed using the methodology.
• The task was extended to examine the rigidity of systems built using the methodology.
• Issues illuminated by these activities have been explored by considering them further, in particular
by extrapolating them to much more complex systems.
• Two users who were previously unfamiliar with BOD have provided feedback on their experi-
ences in using it.
The work has brought to light several areas for consideration including:
• Extending the BOD methodology to accommodate all of the software development life cycle
• Extending the methodology to make it much easier to use for the development of very large and
complex systems
• The potential for adding features to POSH to allow it to better seek multiple goals simultaneously
• A need to improve the ability to test and verify POSH plans to allow BOD to be used with more
confidence in complex systems
Some of the work has produced definite recommendations, but each of the above points has also
raised the possibility of further work well beyond the scopeof a simple evaluation.
94
7.2 Evaluation of the work
7.2.1 Was the best method used for evaluating BOD?
When the evaluation was planned it was expected that it would largely be a programming task. The plan
saw the production of a large amount of software, the development of which would produce ideas and
improvements, the number of which would increase with the amount of code produced. The more lines
of code that are generated, the more exposure to BOD and the more data for the evaluation.
Whilst this plan was followed for the first half of the project it has become clear that it was not a big
programming project, but a big thinking project.
7.2.2 The task
Handling cubes was a good task because it was a simple, practical goal that was easily extensible.
However it is predictable – the cube arrives on the table, therobot sees it, picks it up and processes
it according to the colour. A less predictable task may have provided move opportunities for the robot
to react to unexpected circumstances.
A side effect of the predictability was that there was littlepossibility of any larger scale emergent
behaviour occurring, which may have provided useful information for the evaluation. This is not a
major issue, because emergent behaviour is not necessarilydesirable since it can be very unpredictable
and the timescale for the project was too short to allow detailed investigation. If a robot is carrying out
a specified task then designing emergent behaviour in may be difficult and if it happens accidently then
it might lead to unpredictability and a failure to carry out the task. The task was suited to Strict POSH
– it would have been useful to look at Scheduled POSH too.
It might have been better to more closely define what sort of robotic applications BOD was be-
ing evaluated for. There is a big difference between the highly specialised tasks that most robots are
designed for, and the loosely defined and wider ranging tasksthat robots are used for in research.
The definition of “robot” is
1. (a) machine that looks like a human being and performs various complex acts (as
walking or talking) of a human being; also : a similar but fictional machine
whose lack of capacity for human emotions is often emphasised
(b) an efficient insensitive person who functions automatically
2. a device that automatically performs complicated often repetitive tasks
3. a mechanism guided by automatic controls
[Merriam-Webster Dictionary]
Most people think of the first of these definitions when considering robots, and see them as experi-
mental devices which will only become widespread in the distant future. However robots are common-
place devices, since the majority fall into the last two definitions. The work has largely focussed on the
second definition, picking up cubes from a table is a complicated but repetitive task since this is the type
of activity that the majority of robots perform.
95
More emphasis could have been put onto the third definition since this could have experimented
more with the robot’s advantage to cope with the unexpected.The third definition really also cover
many electronic devices containing embedded computers such as mobile telephones and home automa-
tion systems since these devices contain software mechanisms for prioritising actions and guiding their
behaviour. However this straying away from “classical” robotics and into control systems, but there is
clearly scope to use BOD in these devices.
Could a simpler task which would have avoided the problems with kinematics and grasping, but still
provide the same information about BOD? As part of the preliminaries for this project, a very simple
POSH plan for raising the robot’s arms was implemented usingYARP’s own interface to Python. This
exercise showed that a more complex task would be necessary to evaluate BOD, and that YARP’s own
Python interface was inadequate for that task.
7.2.3 jyPOSH engine
Using the jyPOSH engine had both advantages and disadvantages. The advantages were that no time
need be spent developing a POSH engine since it was readily available and relatively easy to integrate
with the iCub software. The engine implemented POSH – writing a new engine might have deviated
from POSH in some way and so the evaluation would have been of something slightly different.
However, the documentation of the engine is not perfect and it is difficult to understand some of its
features and it is missing features of other POSH implementations such as the ability to re-use parts of
the plan on multiple behaviour objects (as in the case of the sonar elements in the Nomad 200). It may
have been better to have used Joanna Bryson’s earlier C++ engine since the iCub and YARP themselves
are written in C++.
The jyPOSH engine is not easy to modify because of the requirement for backward compatibility
for the existing applications unless a new branch was created. Moving away from .lap files would have
been beneficial since it would allow a considerable range of changes to be experimented with.
7.2.4 iCub robot
The choice of an iCub was not ideal – it restricted development for practical purposes to using a sim-
ulator since access to a real robot was very limited. Howeverthe availability of sophisticated robots is
very limited, so a simulator would probably have been used whatever platform had been chosen.
On the other hand the iCub was a good choice because it is a verycomplex robot and thus a good
platform to use. One alternative would have been to not use a robot in the normal sense (definition
above) but used a complex automated device.
A mobile robot might have been better, since it would have more capacity for encountering unex-
pected situations. It is likely that the advantages of BOD, in particular POSH, would have come through
very strongly in such an environment.
However this is close to the original Nomad 200 work, and may not have shown up anything new.
It was nevertheless very useful the iCub has several complexindependent subsystems as this has high-
lighted several issues, particularly the ability to perform multiple tasks simultaneously.
96
7.2.5 Real robot
Using a real robot for the development, if it had been practical, may have added further complication un-
related to BOD. Whilst there would have been little difficultywith grasping other mechanical difficulties
may have swamped the problem.
The simulator is adequate to answer my question. However a simulator is often seen as very much
the second best to using real hardware.
The lack of a real robot did not prevent various issues from being identified but other issues may
have been raised by it. An iCub was a good choice because its complexity forces issues such as multiple
goals to the fore. The simulator caused various problems, although it is not generally used for the type
of tasks that I was using it for.
The differences between the real robot and the simulator areprobably far less significant than be-
tween two different types of robot, although there are conceivably special issues in the real robot which
might impact the BOD methodology. However porting the code to a real iCub would dramatically
demonstrate that BOD is a practical methodology for such work.
The only issue which caused real problems because of the simulator was grasping; in the real robot
this would not have been such a big problem, but vision processing, for example, may have proved
problematic instead.
A compromise might have been to port a version of the code to the robot halfway through the project
and then continued with the simulator. A difficulty with trying to maintain code which runs on both is
that it would need to be regularly checked on a real robot.
7.2.6 Learning
This thesis did not consider how well BOD facilities learning in a robotic environment. Learning is
important in modern artificial intelligence because it allows the agent to grow beyond its original pro-
gramming. Many tasks are impossible to specify in advance and it is easier for the agent to adjust its
internal state to compensate for its environment.
Learning was excluded as it potentially added greatly to thetask. At its most primitive level, learn-
ing is a slow feedback mechanism, and I felt it could safely beignored since other, faster, feedback
mechanisms are present in the code. The only area where it wasseriously considered was whether to
modify the code which moves the hand to the cube in small steps, to adjust the step size.
7.2.7 Other issues
Some of the other issues cost time – notably difficulties withgrasping and inverse kinematics. These
would have been reduced with a different platform – in particular the simulator’s poor handling of grasp-
ing and difficulties in accessing a real robot would have beenavoided with a different robot. However
it is not obvious that access to any other type of sophisticated robot would have been any easier.
More effort could have been spent on looking at testing of POSH plans, and tailoring BOD to be
better suited for testing.
97
7.3 How the project plan evolved
The original project plan is discussed in Section 4.1. The scope of the project reduced in three respects:
1. The scale of second task was reduced from that originally envisaged. There is a trade-off between
the complexity of the task, the details of which risk overwhelming the evaluation of BOD and
choosing a sufficiently complex task to fully exercise BOD. For example a task originally envis-
aged was tossing an object in the air and catching it, but the dynamics and kinematics involved
would have dominated the exercise.
2. Porting to a real robot was not done, for reasons explainedin Section 7.2.5.
3. The evaluations by other people were scaled back partly because sufficient information could be
gathered from a limited exercise and partly because of unanticipated work such as modifications
to the simulator.
98
Chapter 8
Conclusions
Behaviour Oriented Design worked well for the tasks undertaken for this thesis, and allowed a relatively
simple but functional system to be built quickly and easily,and then extended without difficulty. The
difficulties encountered in the project were unrelated to BOD, and the iterative methodology was helpful
in overcoming them.
Nevertheless there are opportunities to improve it, particularly to support its use in large and complex
systems:
• The BOD methodology can be extended to cover the full software development life cycle.
• Small changes in the specification of state elements and the initial list of primitives may avoid
problems in later iterations of the development cycle.
• POSH can be extended to automatically detect and pursue multiple non-conflicting goals.
• POSH can be extended to cleanly abort the execution of a driveelement.
• A recommendation could be added that chained competences should always be clearly marked as
such.
• Mechanisms can be introduced to make POSH plans more modularto help in larger systems.
• Work to improve the verification and testing of POSH plans would be of considerable benefit.
8.1 Future work
This project has suggested several areas for further work.
8.1.1 Extending the evaluation project
• Porting the code to a real iCub to investigate whether there are further BOD-related issues.
99
• Evaluating the effectiveness of BOD within an environment where the agents perform a consid-
erable amount of learning, for example by adjusting the threshold at which triggers operate.
• Evaluating BOD for the implementation of a complex consumer-oriented, real-time multi-tasking
system which pursues multiple goals.
8.1.2 Extending the BOD methodology
• Extending POSH to detect and simultaneously pursue multiple non-conflicting goals.
• Extending BOD to improve its coverage of requirements definition and testing.
• Developing strategies for verification and automated testing of POSH plans.
8.1.3 Areas other than BOD or its evaluation
• Developing an improved, more task-oriented, way of representing kinematics in Cartesian space.
• Developing an improved language for implementing behaviours within a robotic programming
environment, particularly in respect of kinematics.
• Developing a more rigorous understanding of benefits and costs of producing very high fidelity
robotic simulators validated against real hardware.
100
Bibliography
P. Amestoy, et al. (2002). ‘MUMPS: A Multifrontal MassivelyParallel Solver’. ERCIM News
(50):679–698.
A. Basu, et al. (2008). ‘Incremental Component-Based Construction and Verification of a Robotic
System’. InProceedings of the 18th European Conference on Artificial Intelligence, ECAI 2008, vol.
178 ofFrontiers in Artificial Intelligence and Applications, pp. 631–635. IOS Press.
M. Billinghurst, et al. (2010). ‘ARToolKit’. http://www.hitl.washington.edu/
artoolkit/documentation/ .
B. Boehm (1986). ‘A spiral model of software development andenhancement’.SIGSOFT Softw. Eng.
Notes11(4):14–24.
R. P. Bonasso, et al. (1997). ‘Experiences with an architecture for intelligent, reactive agents’.Journal
of Experimental & Theoretical Artificial Intelligence9(Issue 2–3):237–256.
G. Bradski & A. Kaebler (2008).Learning OpenCV - Computer vision with the OpenCV Library. O’
Reilly.
C. Breazeal & B. Scassellati (1999). ‘How to build robots andinfluence people’. InProceedings of the
1999 IEEE/RSJ International Conference on Intelligent RObots and Systems IROS (1999), Kyongju,
Korea, vol. 2, pp. 858–863.
C. Brom, et al. (2006). ‘POSH Tools for Game Agent Development by Students and Non-Programmers’.
In Q. Mehdi, F. Mtenzi, B. Duggan, & H. McAtamney (eds.),Proceedings of the 9th International
Computer Games Conference: AI, Mobile, Educational and Serious Games, pp. 126–133. University
of Wolverhampton.
F. P. Brooks (1995).The Mythical Man-Month – 20th Aniversary Edition. Addison Wesley.
R. A. Brooks (1986). ‘A Robust Layered Control System for a Mobile Robot’.IEEE Journal of Robotics
and Automation2:14–23.
R. A. Brooks (1989). ‘A Robot that walks: Emergent Behaviourfrom a Carefully Evolved Network’.
Neural Computation1(2):253–262.
R. A. Brooks (1990). ‘Elephants Don’t Play Chess’.Robotics and Autonomous Systems6:3–15.
101
R. A. Brooks (1991). ‘Intelligence without Reason’. InProceedings of the 12th International Joint
Conference on Artificial Intelligence, pp. 139–159.
R. A. Brooks (1992). ‘Artificial Life and Real Robots’. In F. J. Varela & P. Bourgine (eds.),Toward a
Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life,
pp. 3–10, Cambridge, MA, USA. MIT Press.
H. Bruyninckx (2001). ‘Open robot control software: the OROCOS project’. InProceedings 2001
ICRA. IEEE International Conference on Robotics and Automation, vol. 3, pp. 2523–2528. IEEE.
J. Bryson, et al. (2000). ‘Hypothesis Testing for Complex Agents’. InNIST Workshop on Performance
Metrics for Intelligent Systems.
J. J. Bryson (2001a).Intelligence by Design: Principles of Modularity and Coordination for Engineer-
ing Complex Adaptive Agents. Ph.D. thesis, Massachusetts Institute of Technology.
J. J. Bryson (2001b). ‘The Study of Sequential and Hierarchical Organisation of Behaviour via Artificial
Mechanisms of Action Selection’. Master’s thesis, University of Edinburgh.
J. J. Bryson (2007). ‘Mechanisms of Action Selection: Introduction to the Special Issue’.International
Society for Adaptive Behavior15(1):1–5.
J. J. Bryson (2010). ‘Advanced Behavior Oriented Design Environment (ABODE)’. http://www.
cs.bath.ac.uk/\ ˜ {}jjb/web/BOD/abode.html .
J. J. Bryson, et al. (2006). ‘Integrating Life-Like Action Selection into Cycle-Based Agent Simulation
Environments’. InProceedings of Agent 2005: Generative Social Processes, Models, and Mecha-
nisms, pp. 67–81.
J. J. Bryson & J. Flack (2002). ‘Emotions and Action Selection in an Artificial Life Model of Social Be-
havior in Non-Human Primates’. InProceedings of the International Workshop on Self-Organization
and Evolution of Social Behaviour, Monte Verita, pp. 42–45.
J. J. Bryson & J. C. S. Leong (2007). ‘Primate errors in transitive ‘inference’: a two-tier learning model’.
Animal Cognition10(1):1–15.
J. J. Bryson, et al. (2002). ‘Toward behavioral intelligence in the Semantic Web’.Computer35(11):48–
54.
J. J. Bryson, et al. (2003). ‘Agent-Based Composite Services in DAML-S: The Behavior-Oriented
Design of an Intelligent Semantic Web’. In N. Zhong, J. Liu, &Y. Yao (eds.),Web Intelligence, pp.
37–56. Springer-Verlag.
J. J. Bryson & B. McGonigle (1997). ‘Agent Architecture as Object Oriented Design’. InProceedings
of the 4th International Workshop on Agent Theories, Architectures, and Languages (ATAL97), pp.
15–30. Springer-Verlag.
102
J. J. Bryson & L. Stein (2001). ‘Modularity and Design in Reactive Intelligence’. InProceedings of the
17th international joint conference on Artificial intelligence, vol. 2, pp. 1115–1120.
J. J. Bryson & K. R. Thorisson (2001). ‘Dragons, Bats & Evil Knights: A Three-Layer Design Approach
to Character Based Creative Play’.intelligent Virtual Agents5(2):57–71.
J. Canny (1986). ‘A Computational Approach To Edge Detection’. IEEE Transactions on Pattern
Analysis and Machine Intelligence8(3):679–698.
Y. M. Chen & S. H. Chang (2008). ‘An agent-based simulation for multi-UAVs coordinate sensing’.
International Journal of Computing and Cybernetics1(2):269–284.
A. Chopra, et al. (2006). ‘The Nomad 200 and the Nomad SuperScout: Reverse engineered and res-
urrected’. InProceedings of the 3rd Canadian Conference on Computer and Robot Vision table of
contents, pp. 55–62. IEEE Computer Society.
A. Clark (1999). ‘An Embodied Cognitive Science?’.Trends In Cognitive Sciences3(9):345–351.
J. Connell (1989a). ‘A behavior-based arm controller’.IEEE Transactions on Robotics and Automation
5(6):784–791.
J. Connell (1989b). ‘A Colony Architecture for an ArtificialCreature’. Tech. Rep. Artificial Intelligence
Technical Report 1151, Massachusetts Institute of Technology.
A. P. del Pobil (2006). ‘Why do We Need Benchmarks in Robotics Research?’. In Lecture Notes for
IROS 2006 Workshop II (WS-2).
A. D’Souza, et al. (2001). InProceedings of the IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS 2001), Maui, USA, vol. 1, pp. 298–303.
T. Elrad, et al. (2001). ‘Aspect-oriented programming: Introduction’. Communications of the ACM
44(10):29–32.
EURON (2010). ‘Benchmarking Initiative - Manipulation andGrasping’. http://www.euron.
org/activities/benchmarks/grasping .
P. Fitzpatrick, et al. (2008). ‘Towards Long-Lived Robot Genes’. Robotics and Autonomous Systems
56(1):29–45.
E. Gamma, et al. (1995).Design Patterns: elements of reusable object-oriented software. Addison-
Wesley Longman Publishing Co., Inc., Boston, MA, USA.
E. Gat (1998). ‘On three-layer architectures’. In D. Kortenkamp, P. R. Bonnasso, & R. Murphy (eds.),
Artificial Intelligence and Mobile Robots, pp. 195–210.
J. Gemrot, et al. (2009). ‘Pogamut 3 Can Assist Developers inBuilding AI (Not Only) for Their
Videogame Agents’. InAgents for Games and Simulations: Trends in Techniques, Concepts and
Design, vol. 5920 ofLecture Notes in Computer Science, pp. 1–15. Springer.
103
I. D. Horswill (1995). ‘Specialization of Perceptual Processes’. Tech. Rep. AITR-1511, Massachusetts
Institute of Technology.
L. Hugues & N. Bredeche (2006). ‘Simbad: An Autonomous RobotSimulation Package for Education
and Research’. In S. Nolfi, G. Baldassarre, R. Calabretta, J.Hallam, D. Marocco, J.-A. Meyer,
O. Miglino, & D. Parisi (eds.),From Animals to Animats 9, vol. 4095 ofLecture Notes in Computer
Science, pp. 831–842. Springer Berlin / Heidelberg.
J. Jesse (2009). ‘A kinematic model for the iCub’. Tech. Rep.Semester project, Ecole Polytechnique
Federal De Lausanne.
N. M. Josuttis (1999).The C++ Standard Library: A Tutorial and Reference. Addison-Wesley.
G. A. Kaminka, et al. (2002). ‘GameBots: a flexible test bed for multiagent team research’.Communi-
cations of the ACM45(1):43–45.
E. Kapellos, et al. (1995). ‘Formal Verification in Robotics: Why and How?’. Inin The Interna-
tional Foundation for Robotics Research, editor, The Seventh International Symposium of Robotics
Research, pp. 20–1. Press.
H. Kitano, et al. (1997). ‘RoboCup: A challenge problem for AI and robotics’. InRoboCup-97: Robot
Soccer World Cup I, vol. 1395 ofLecture Notes in Computer Science, pp. 1–19. Springer-Verlag.
A. Koestler & T. Braunl (2004). ‘Mobile Robot Simulation with Realistic ErrorModels’. In ICARA
2004; The second International Conference on Autonomous Robots and Agents.
T. J. Koo, et al. (1999). ‘A Formal Approach to Reactive System Design: Unmanned Aerial Vehicle
Flight Management System Design Example’. InIEEE International Symposium on Computer-Aided
Control System Design, Kohala.
G. Kragten & J. Herder (2010). ‘A Platform for Grasp Performance Assessment in Compliant or Un-
deractuated Hands’.Mechanism and Machine Theory132(2).
S. Kumar & R. Mishra (2008). ‘Semantic Web Service Composition’. IETE Technical Review25(3).
A. Kwong (2003). ‘A Framework for Reactive Intelligence through Agile Component-Based Behav-
iors’. Master’s thesis, University of Bath.
C. Laird, et al. (2006). ‘Introduction to IPOPT: A tutoroal for downloading, installing and using IPOPT’.
Tech. Rep. Revision 799.
C.-U. Lim, et al. (2010). ‘Evolving Behaviour Trees for the Commercial Game DEFCON’. InAppli-
cations of Evolutionary Computation, vol. 6024 ofLecture Notes in Computer Science, pp. 100–110.
Springer.
S. Luke, et al. (2005). ‘MASON: A Multiagent Simulation Environment’.Simulation81(7):517–527.
Z. Macura, et al. (2009). ‘A Cognitive Robotic Model of Grasping’. In Ninth International Conference
on Epigenetic Robotics (EPIROB 09), Venice, Italy.
104
P. Maes (1991a). ‘The agent network architecture (ANA)’.ACM SIGART Bulletin2(4):115–120.
P. Maes (1991b). ‘Behavior Based Artificial Intelligence’.In Proceedings of the 15th annual conference
of the Cognitive Science, pp. 74–83.
J. Maitin-Shepard, et al. (2010). ‘Incremental Component-Based Construction and Verification of a
Robotic System’. In2010 IEEE International Conference on Robotics and Automation (ICRA), pp.
2308 – 2315. IEEE.
R. Manseur (2006).Robot modeling and Kinematics. Da Vinci Engineering Press.
M. J. Mataric (1997). ‘Behavior-Based Control: Examples from Navigation, Learning, and Group
Behavior’. Journal of Experimental and Theoretical Artificial Intelligence, special issue on Software
Architectures for Physical Agents9(2–3):323–336.
G. Metta, et al. (2006). ‘YARP: Yet another Robot Platform’.International Journal of Advanced
Robotics Systems, special issue on Software Development and Integration in Robotics3(1):41–48.
G. Metta, et al. (2010). ‘Project No. 004370 - D2.1 A Roadmap for the Development of Cognitive
Capabilities in Humanoid Robots’. Tech. Rep. Draft 1.0, University of Genoa, DIST – LIRA-Lab.
G. Metta, et al. (2008). ‘The iCub humanoid robot: an open platform for research in embodied cog-
nition’. In Proceedings of PerMIS: Performance Metrics for Intelligent Systems Workshop. August
19-21, 2008, Washington DC, USA.
O. Michel & F. Rohrer (2008). ‘Rat’s Life: A Cognitive Robotics Benchmark’. In8th Workshop
on Performance Metrics for Intelligent Systems (PerMIS ’08), Gaithersburg, Maryland, vol. 44 of
Springer Tracts in Advanced Robotics, pp. 43–49. ACM.
G. A. Miller (1956). ‘The magical number seven, plus or minustwo: some limits on our capacity for
processing information. 1956.’.Psychological Review63:81–97.
N. Mohamed, et al. (2008). ‘Middleware for Robotics: A Survey’. In 2008 IEEE Conference on
Robotics, Automation and Mechatronics, Chengdu, pp. 736–742. IEEE.
S. R. Palmer & J. M. Felsing (2002).A Practical Guide to Feature-Driven Development. Prentice Hall.
M. Paolucci & K. Sycara (2003). ‘Autonomous Semantic Web services’. IEEE Internet Computing
7(5):34–41.
S. J. Partington & J. J. Bryson (2005). ‘The Behavior Oriented Design of an Unreal Tournament Char-
acter’. InProceedings of the 5th International Workshop on Intelligent Virtual Agents, IVA’05, vol.
3661 ofLecture Notes in Artificial Intelligence, pp. 466–477. Springer-Verlag.
C. Pepper, et al. (2007a). ‘Robot simulation physics validation’. In PerMIS ’07: Proceedings of the
2007 Workshop on Performance Metrics for Intelligent Systems, pp. 97–104, New York, NY, USA.
ACM.
105
C. Pepper, et al. (2007b). ‘Robot simulation physics validation’. In PerMIS ’07: Proceedings of the
2007 Workshop on Performance Metrics for Intelligent Systems, pp. 97–104, New York, NY, USA.
ACM.
T. Perkins (2006). ‘Technical Approach of the End to End Deployment Simulation (E2EDS)’. In2006
Fall Simulation Interoperability Workshop, 13 September 2006.
P. Pirjanian (2000). ‘Multiple objective behavior-based control’. Robotics and Autonomous Systems
31(1-2):53–60.
T. J. Prescott, et al. (1999). ‘Layered Control Architectures in Robots and Vertebrates’.Adaptive
Behavior7:99–127.
M. Proetzsch, et al. (2007). ‘The Behaviour-Based Control Architecture iB2C for Complex Robotic
Systems’. InProceedings of the 30th annual German conference on Advances in Artificial Intelli-
gence, vol. 4667 ofLecture Notes in Computer Science, pp. 494–497. Springer-Verlag.
M. Proetzsch, et al. (2010). ‘Development of complex robotic systems using the behavior-based control
architecture iB2C’. pp. 46–67.
D. Rebuzzi, et al. (2006). ‘Muon Detector-Description as-built and its Simulation for the ATLAS Ex-
periment’. InNuclear Science Symposium Conference, no. 3, pp. 1647–1651. IEEE.
P. J. Richards (2004). ‘An Integrated Development Environment for Behaviour Oriented Design’. BSc
dissertation, University of Bath.
A. Rimoldi, et al. (2004). ‘The simulation for the ATLAS experiment. Present status and outlook’. In
Nuclear Science Symposium Conference, no. 3, pp. 1886–1890. IEEE.
J. K. Rosenblatt (1997). ‘DAMN: A Distributed Architecturefor Mobile Navigation’. Journal of
Experimental & Theoretical Artificial Intelligence9(2–3):339–360.
J. K. Rosenblatt & D. Payton (1989). ‘A fine-grained alternative to the subsumption architecture for
mobile robot control’. InProceedings of the IEEE/INNS International Joint Conference on Neural
Networks.
W. W. Royce (1970). ‘Managing the Development of Large Software Systems: Concepts and Tech-
niques’. InTechnical Papers of the IEEE Western Electronic Show and Convention (WesCon 26).
IEEE.
D. C. Schmidt, et al. (1993). ‘ADAPTIVE: A dynamically assembled protocol transformation, integra-
tion and evaluation environment.’.Concurrency: Practice and Experience5(4):269––286.
A. Schmitz, et al. (2008). ‘A Prototype Fingertip with High Spatial Resolution Pressure Sensing for the
Robot iCub’. In8th IEEE-RAS international Conference on Humanoid Robots, pp. 423–428.
R. Smith (2006).Open Dynamics Engine (ODE) v0.5 User Guide.
106
I. Sommerville (2006).Software engineering (8th ed.). Addison Wesley Longman Publishing Co., Inc.,
Redwood City, CA, USA.
Swig (2009). ‘SWIG-1.3 Documentation’.http://www.swig.org/Doc1.3/index.html .
K. R. Thorisson (1999). ‘A Mind Model for Multimodal CommunicativeCreatures & Humanoids’.
International Journal Of Applied Artificial Intelligence13(4-5):449–486.
V. Tikhanoff, et al. (2008). ‘The iCub humanoid robot simulator’. In Proceedings of the 2008 IEEE/RSJ
International Conference on Intelligent RObots and Systems (IROS 2008), Nice, France, pp. 22–26.
M. Torrance (1992). ‘The Case for a Realistic Mobile Robot Simulator’. In in Working Notes of the
AAAI Fall Symposium on Applications of Artificial Intelligence to Real-World Autonomous Mobile
Robots, pp. 23–25.
N. Tsakarakis, et al. (2007). ‘iCub – The Design and Realization of an Open Humanoid Platform for
Cognitive and Neuroscience Research’21(10):1151–1175.
T. Tyrell (1993). Computational Mechanisms for Action Selection. Ph.D. thesis, University of Edin-
burgh.
D. Vernon, et al. (2007). ‘The iCub Cognitive Architecture:Interactive Development in a Humanoid
Robot’. In Proceedings of the 6th International Conference on Development and Learning (ICDL
2007), pp. 122–127.
F. M. Wahl & U. Thomas (2002). ‘Robot Programming - From Simple Moves to Complex Robot
Tasks’. InFirst International Colloquium ’Collaborative Research Centre 562’ - Robotic Systems for
Modelling and Assembly’, pp. pages 245–259.
T. Wimbock, et al. (2007). ‘Impedance Behaviors for Two-handed Manipulation: Design and Experi-
ments’. InInternational Conference on Robotics and Automation, ICRA2007, pp. 4182–4189. IEEE.
M. Wojtczyk & A. Knoll (2008). ‘A Cross Platform DevelopmentWorkflow for C/C++ Applications’.
Software Engineering Advances, International Conferenceon0:224–229.
C. Zielinski, et al. (2006). ‘Rubik’s cube puzzle as a benchmark for service robots’. In12th IEEE
International Conference on Methods and Models in Automation and Robotics, MMAR, pp. 579–
584. IEEE.
107
Appendix A
Requirements
This section examines the overall requirements for the evaluation task.
A.1 Requirements Analysis
Non-production code which is being developed as a research project has by its nature high-level and
broad requirements. The scope and direction of the implementation is not necessarily clear at the start,
and so requirements are likely to change during the work. It is expected that the library will be published
to be available for other projects and consequently needs tobe robust.
A.1.1 Goals
The goal of the software project was to implement some simpleprototype functionality on a robot,
and subsequently to extend it. The purpose of this is not to produce an application, but to use the
development of one as a vehicle to evaluate the BOD methodology.
The software needs to provide a platform which can be used to experiment with BOD in a robotic
setting. It needs to be sufficiently flexible and easy to use that different scenarios can be constructed
by assembling POSH plans and simple behaviours. There should be no need to undertake complex low
level programming of the robot since the toolkit is a framework in which ideas can be tried out.
The methodology to be used is itself a clear requirement since the goal of the work is to assess it.
The task to be implemented should not be technically difficult but should be complex enough that it
exposes the BOD methodology to real problem solving.
A.1.2 Platform
The robot needs of be highly sophisticated with a rich and complex interface to provide a model for
modern robots, and to allow BOD to be used in a demanding environment. It needs to have a simulator
since developing all code on a real robot would require constant physical access to the robot and make
the project vulnerable to hardware failures. The availability of any type of complex robot is rather
limited.
108
An iCub is a very complex robot suitable for the task. Expertise is readily available in the form of an
active and interconnected user community. It uses YARP, a modern robot communications layer, which
is representative of modern systems. For this the software needs to be able to communicate with the
iCub, through YARP, and with OpenCV for vision processing.
The software ideally needs to run on a real iCub robot. However there are only four robots in the UK
and access is likely to be limited. The closest are around twohours from Bath, so a one-day visit will
involve four hours travelling. Even then access is likely tobe be limited because of other users. There
are other opportunities–the summer school will have two robots, but with 50 people attending access to
the robot is again likely to be limited. The code therefore needs to run on the simulator, but with the
possibility to be ported to a real robot. This impacts the choice of task, since for example crawling puts
a great strain on the robot and walking is not possible with the unmodified feet.
A.1.3 POSH engine
There is a need to use a POSH engine, and implementations are already available in several languages.
However the choice quickly falls to two – using a C++ one or onein a scripting language. The benefit of
using C++ is that interfacing to the iCub software will be relatively straightforward, whilst a scripting
language will help make development faster since prototypesystems are more quickly developed in
such languages. Bryson wrote a C++ version in the late 1990s and much work has been done since
using the Python POSH engine jyPOSH. However the short timescales of the project suggested that the
scripting language route would be best.
The underlying YARP software is available as a minimal Python library which could be extended,
or a separate wrapper could be written to make the iCub more accessible using calls to its API, also in
C++. Some help is needed to make the underlying YARP and iCub software easy to use from Python.
A.1.4 Task
The task needs to be reasonably complex and use more than one of the robots sub-systems. It must
involve feedback from sensors since reactive planning depends upon this. Vision is an appropriate sense
since it is reasonably complex but its wide use means that there are well known techniques readily
available in software. Since the lower body of the robot is not available a task involving the hands
is most appropriate. The complex arm on an iCub will also provide experience in coordinating the
operation of multiple joints, ideally lifting an object. Nevertheless the task needs to be clearly defined
otherwise there is the risk that the methodology is not followed to produce a goal, but the goal is subtly
altered because of problems with the methodology.
A.1.5 Development
The robot needs to have a good low level library so that time does not have to be spent writing low level
drivers, and it should be possible to incorporate good logging for debugging. The task should be capable
of being left running for on extended period for testing purpose but not be of such lengthy duration that
it is hard to quickly test any changes. Consequently a cyclicoperation is preferred.
109
A.2 Requirements specification
Requirement Source
Software needs to communicate with the iCub A.1.2
Software needs to monitor responses from iCub A.1.2
Software needs to contain adequate debugging facilities A.1.5
Software needs to be modular A.1.4
Software needs to be runnable (eventually) on real iCub A.1.2
Software needs to be developed using BOD A.1.1
Software needs to use a POSH engine A.1.1
Software needs a reactive planning element A.1.1
Software needs to interface to OpenCV A.1.2
Software needs to interface to YARP A.1.2
Software needs to be testable A.1.5
Software needs to be capable of being well defined A.1.4
Software needs to be able to be ported to the real robot A.1.2
It has to work with the simulator. Very limited access to robot A.1.2
The robot should be able to see the object A.1.4
The robot should be able to move its hands to the object A.1.4
The robot should be able to grasp the object A.1.4
The robot should be able to lift the object A.1.4
The robot should be able to drop the object A.1.4
The robot should be able to move without colliding with the table, the
object or its own body
A.1.4
The operation should be cyclic to assist debugging A.1.5
It should be possible to film part of the operation A.1.5
Suitable log files should be written A.1.5
Software needs to be flexible, allowing changes at the top level A.1.1
Need a high level toolkit from which functionality can be assembled
without being distracted by detail
A.1.1
110
Appendix B
Using the software
B.1 Installing the software needed to run the simulated tasks
The iCub software is supported on Windows XP and Linux (Debian and Ubuntu). The work undertaken
for this project was done exclusively on Ubuntu, and becauseof time limitations the software developed
for it has only been tested on this operating system.
The code has been demonstrated to run on X8664 Ubuntu 9.10 and Ubuntu 10.04. The code was
also tested on an IA32 Ubuntu 10.4 system. The following software packages and their dependencies
need to be installed from the Ubuntu repository to compile and run the iCub software including the
simulator and jyPOSH. The dependencies are automatically installed on Ubuntu when these packages
are installed usingapt-get install .
• cmake-curses-gui . Strictly speaking onlycmake andcmake-data are needed but this
gives a GUI interface to CMakeLists.txt
• subversion . Subversion code management system
• libace-dev . ACE networking library
• gfortran . Fortran 95 compiler and run-time library needed for IPOPT and associated packages
• g++. Version 4.4.3-4 was used for the work described here
• libgsl0-dev . GNU Scientific Library
• libcv-dev . OpenCV core
• libcvaux-dev . OpenCV auxiliary packages
• libhighgui-dev . OpenCV GUI. this is strictly speaking not needed but prevents many of the
iCub software components from failing to build
• libsdl-dev . Simple DirectMedia Layer
111
• libglut3-dev . OpenGL graphics
• swig1.3 needed by the jyPOSH iCub code
• python2.6-dev needed by SWIG when building C++/Python interface layers
• doxygen
The following packages need to be downloaded and installed from source code. At the time of
writing ODE is not included in the Ubuntu repository andcoinor-ipopt containing the IPOPT
package does not work because it does not contain MUMPS or theHarwell Software Library. The
versions used are the latest at the time of writing. These packages are all downloaded and built from a
normal account since they are not installed in system directories –sudo access is not needed for them
although it obviously is to install the above Ubuntu packages.
B.2 ODE 0.11.1
ODE 0.11.1 needs to be built with double precision enabled. Download from
https://sourceforge.net/projects/opende/files
Unpack and run the commands:
. / c o n f i g u r e −−enab le−double−p r e c i s i o n
make
B.3 IPOPT 3.8.3
Download the source code from
https://www.coin-or.org/download/source/Ipopt/Ipopt -3.8.3.tgz
and unpack. Install BLAS and MUMPS by changing the current directory to Ipopt-3.8.3/ThirdParty.
Change to the Blas sub-directory and run the Bash scriptget.Blas to download it from netlib.org.
Build it by running the command./configure followed by the commandmake.
Change to the Mumps sub-directory and run the Bash scriptget.Mumps to download MUMPS
from mumps.enseeiht.fr. Once it has been downloaded build it using the command./configure
followed bymake.
As an alternative to Mumps you may choose instead to install the Harwell Software Library files
ma27ad.f and mc19ad.f into Ipopt-3.8.3/ThirdParty/HSL, which are the solvers recommended by the
iCub developers. This is especially true if you are using some of the other code in the iCub which uses
Ipopt, for example the iDyn force control library which is not used, and thus not tested, as part of this
work. To obtain these register to use the HSL Archive at http://www.hsl.rl.ac.uk/archive/hslarchive.html.
Note that there are restrictions on the use of software from the archive. Build the object code by running
112
./configure followed by make. You can do both – the HSL code will be used in preference to
MUMPS if you do so.
Although BLAS is compiled as part of this process the jyPOSH code will actually use the version
bundled with OpenCV. However it is necessary to configure BLAS into IPOPT otherwise it won’t build.
Once the third party packages that you need are compiled, change the current directory to Ipopt-3.8.3
and run the commands:
. / c o n f i g u r e
make i n s t a l l
The install parameter merely causes the shared library file to be installed into Ipopt-3.8.3/lib/
directory.
The reader is also advised to read that portion of the iCub documentation on installing the library,
which is available at http://eris.liralab.it/wiki/Installing IPOPT.
B.4 YARP
Download using the command
svn co h t t p s : / / yarp0 . svn . s o u r c e f o r g e . n e t / s v n r o o t / yarp0 / t r u n k / yarp2
A suitable revision shown in Section B.11, so if problems areexperienced with the latest ver-
sion in the repository then remove the yarp2 directory created and repeat the command specifying
-r version after theco . However this will probably require that the specific revision of the iCub
software and Simfork simulator indicated below is used.
To compile YARP change the current directory to the yarp2 directory. Run the commandcmake
. and then run the CMake GUI (ccmake . ). The first generates the CMakeCache.txt file used to
hold configuration information for CMake from CMakeList.txt and the second command runs a GUI to
allow it to be modified. Enable the creation of the YARP maths library (CREATELIB MATH). Rerun
the configuration (c theng inside ccmake) and then runmake.
B.5 Simfork
The modifications to the simulator are downloaded from Sourceforge using the command
svn co h t t p s : / / s im fo rk . svn . s o u r c e f o r g e . n e t / s v n r o o t / s im fo rk
A suitable revision is shown in Section B.11 if problems are encountered with the latest revision.
The easiest way of building the simulator is to patch it into the iCub code as described below.
It is possible to use the iCub simulator instead of Simfork but it needs modifying. The file main.cpp
needs to be changed so that the “grab” function works when thefull hand is enabled.
B.6 Environment variables
The following environment variables need to be defined.
113
• YARPDIR, to the root of the YARP directory/yarp2
• ICUB ROOT, to the root of the iCub directory/iCub
• PYTHONPATH, to the compiled ICubInterface code/jyposh/jyposh/trunk/library/iCub/iCubInterface/lib
• IPOPT DIR, to IPOPT,/Ipopt-3.8.3
B.7 iCub code
Download the iCub software using the command
svn co h t t p s : / / r o b o t cu b . svn . s o u r c e f o r g e . n e t / s v n r o o t / ro b o t cu b / t r u n k / iCub}
A suitable version is given in the table in Section B.11 if future changes cause problems. This
revision will almost certainly be required if the Subversion revision of YARP shown in that table is
used.
If the Simfork simulator is to be built, which is advised, then the Simfork directory should be
moved toiCub/main/src/simulators . The CMakeLists.txt file in that directory needs to be
modified so that the Simfork code is build. Edit the CMakeLists.txt file and after the lines:
i f ( ICUB USE ODE AND ICUB USE SDL )
a d d s u b d i r e c t o r y ( iCubS imu la t i on )
add the line
a d d s u b d i r e c t o r y ( iCubS imu la t i on )
The iCub software needs YARP, IPOPT and ODE to have been builtand the environment variables
described above to have been defined. Change the directory toiCub/main and runcmake . fol-
lowed byccmake . . SetICUB USEODE, ICUB USEIPOPT ICUB USESDLandICUB USEGUI
all to ON and configure the build as described above for YARP. Set IPOPT DIR to the top level direc-
tory used for IPOPT (probably/Ipopt-3.8.3) andIPOPT LIBRARIES to /Ipopt-3.8.3/lib/libipopt.so.
Run make -k – the-k switch means that the Makefile continues to run if some components do
not build. The above list of packages if not exhaustive for building all of the iCub software so some
components which are not needed for the jyPOSH work will failto build. These can be built if required
by installing further dependencies from the Ubuntu repository.
B.8 jyPOSH
Check out the jyPOSH software using the command
svn co h t t p s : / / j yposh . svn . s o u r c e f o r g e . n e t / s v n r o o t / j yposh
Change the current directory to/jyposh/jyposh/trunk/library/iCub/iCubInterface and run the com-
mandcmake . followed by ccmake . . Set IPOPT DIR and IPOPT LIBRARIES to the same
114
values as used for the iCub software. Runcmake . and thenmake. The code is documented by
Doxygen, and the documentation will be generated in the /./doc directory.∗
The Simfork directory, and the iCub directory in the jyPOSH tree, can be replaced with the ones on
the CD.
B.9 Running the software
• Edit iCub simulator configuration files.
The file$ICUB ROOT/app/simConfig/conf/iCub parts activation.ini controls
which parts of the robot will respond to commands.
/ / / i n i t i a l i z a t i o n f i l e f o r t h e i cub s i m u l a t i o n p a r t s
[SETUP]
e l e v a t i o n o f f
[PARTS]
l e g s on
t o r s o on
l e f t a r m on
l e f t h a n d on
r i g h t a r m on
r i g h t h a n d on
head on
f i x e d h i p on
[ VISION ]
cam on
[RENDER]
o b j e c t s o f f
cove r on
An optional step to improve the performance of theSimfork simulator is to comment out the
line which enables textures and turns on the sky texture (textures sky in the file
app/simConfig/conf/Video.ini .
/ / Uncomment t h e l i n e t h a t s t a r t s w i t h ” t e x t u r e s ” t o d e c l a r ewhat
/ / v i d eo t e x t u r e s you want . You can have s e v e r a l v i d eo t e x t u re s .
/ / Each v id eo t e x t u r e i s a s s i g n e d a t e x t u r e I n d e x , which co r responds
/ / t o t h e t e x t u r e i n d i c e s used i n t h e s i m u l a t o r .
/ / I f you don ’ t uncomment t h e t e x t u r e s l i n e , no p o r t s w i l l be cr e a t e d .
/ / t e x t u r e s sky
[ sky ]
∗The CMake configuration files FindOpenCV.cmake and FindIPOPT.cmake came from the OpenCV Sourceforge project, andRobotcub iCub Sourceforge project respectively
115
t e x t u r e I n d e x 15
p o r t / t e x t u r e
• Start the YARP server
$YARP DIR / b in / yarp s e r v e r
• Start iCub simulator Simfork. Note that this is not part of the iCub distribution but needs to be
added to it as explained in Section B.7.
$ICUB ROOT / main / b in / s im fo rk
• Run jyPOSH
In the directoryjyposh/jyposh/trunk run:
python launch . py−v iCub
B.10 Ancillary code
There are two addition files in iCubInterface/src which may be useful:
• killp.py is a Python script which kills the process running jyPOSH on Linux. The jyPOSH code
contains a control-c handler so this cannot normally be usedto abort it.
• positionTest.cpp is a program which displays the Cartesian coordinates of thehands (and if a
macro is defined, the legs). It is useful as a sanity check thatthe system is working.
B.11 Subversion revisions
A compatible mix of Subversion revisions for the four Sourceforge projects is as follows:
Project Revision
iCub 7081
jyPOSH 807
Simfork 23
YARP 8174
116
Appendix C
Steps needed to port the code to a real
iCub
The following steps are needed to port the cube handling taskfrom the simulator to the real robot:
• Vision processing – object location. The existing algorithm is expected to be OK. If that doesn’t
work then use ARtoolkit.
• Vision processing – hand location. The existing algorithm is expected to be OK but will require a
new template image. If that doesn’t work then use ARtoolkit.
• Velocities – the simulator runs much faster for a given velocity than the real robot, so it will
probably be necessary to increase the velocity in finding thetable.
• Touch sensors – The robot finds the table height by lowering hand onto it (because it has to
fly hand above cube so needs to know heights). The real robot does not have touch sensors in
the fingertips. Will remove this code and hardwire height. Could modify the code to use the
iDyn library to monitor the force sensor in the arm or emulatetouch sensors by partly closing
fingers and using finger position (there is only a tendon opening the finger not closing it – so a
partly closed finger can be opened since it is pushing againsta spring). But pushing arm against
immovable object like a table probably unwise since this could result in large forces on fragile
drive cables. This can be mitigated by using iDyn to provide avirtual spring-damper system in
the arm.
• Change to the Cartesian solver – The Cartesian kinematics solver, which runs as a separate task,
has additional code to prevent the arm from hitting the body.
• Checking for collisions – The kinematics works for the simulator without the hand striking the
body. This will need to be checked with the real robot.
• Grasping – the existing code for grasping will need to be modified to stop when the fingers stop
moving rather than when the touch sensor fires. The fingers aredifferent from the arm inasmuch as
117
they are not driven to close, but do so by the force of a spring,so this is a safe operation to perform.
Alternatively the grasp competence could be replaced with the iCub grasp library module (which
works in the same way), but it would be preferable to continueto use the competence since so
much has been learnt about BOD from it.
• World interface – Replace the code for dropping simulated cubes onto the simulated table to
requests for this to be done manually.
118
Appendix D
Differences between Simfork and
official iCub simulator
The modified simulator is in Sourceforge projectSimfork. Most of the code is the same as in the original
simulator distributed as part of the iCub software but contains the following changes.
All existing simulator commands work in the modified version. Note that the GPL licence on the
original simulator requires that any modifications also have the GPL licence.
• Grab works on all objects regardless of hand type
• Improved resilience – for example program no longer crashesif attempt is made to access non-
existent objects
• Handles a potentially infinite number of objects
• Rationalisation, refactoring and simplification – removing redundant code, redundant variables,
using enumerations instead of numeric constants, factoring out repeated code by moving it into
methods, using inheritance to avoid duplication of structures etc.
• Ability to change the colour of an object. This is useful for testing – cubes which have been
discarded are coloured red.
• Ability to set alpha value as well as RGB values for objects
• Optimisation of routines following profiling
• If all video textures are de-selected in the Video.ini file (typically the linetextures sky com-
mented out) then code for handling the textures is not executed. This gives a dramatic perfor-
mance improvement on a slow computer.
Some problems were identified with the that the modified simulator does not resolve. These have
not not been either fixed because of time constraints and because they did not significantly impact the
BOD evaluation.
119
• The simulators suffer from timeouts in waiting for a joint tomove to a position. In the iCub
software this is implemented as a move followed by a wait for the joint to stop moving. Sometimes
the joint apparently never stops moving because the joint never quite reaches the final desired
position. The solution in the jyPOSH software is to have a timeout of ten seconds and assume
that the joint is in its final position.
• The simulators have problems handling rendering image or video textures onto objects. The
jyPOSH work does not use textured objects so this has not beenpursued.
• The simulators sometimes incorrectly report joint angles.This can cause problems with inverse
kinematics if the current joint angles are specified as well as the final angles, if for example a joint
angle is outside of the permitted range for the joint.
120
Appendix E
Robots versus simulators
During the project the general question of the merits of software simulators compared to real robots was
raised. The robotics research community are in general unenthusiastic about software simulators.
Simple simulators such as Gazebo, USARSim, Player/Stage and Lpzrobots and the iCub simulator
itself are used for a considerable amount of work. However these are not high fidelity simulators and do
not precisely model details of the real environment and so recourse is usually needed to a real robot.
These simulators almost invariably use simple physics modelling packages such as ODE, which
simulate mass and Newtonian dynamics, but not the elasticity of the structures involved and has a very
simple model of the friction between them.
An example of more sophisticated modelling software is the accurate computational contact dy-
namics using finite element analysis for automotive and aerospace design. Similarly the High Energy
Physics community relies on high fidelity simulators (see for instance Rimoldi et al., 2004) – although
the physics simulated is different from that needed in robotics, the level of sophistication is similar. A
key feature of these simulators is validation against real hardware (see for example Rebuzzi et al., 2006).
There are great benefits from using a high fidelity simulator –Bryson et al. (2000) suggests that the
main motivation behind simulators is their cost. A powerfulcomputer, possibly including coprocessors,
is a small fraction of the cost of a highly capable robot. Upgrading the simulated model is unlikely to
require the expense of new hardware, unlike the real robot; and the mechanical fragility of the real robot
is irrelevant.
The arguments against using simulators are also summarisedby Bryson et al. (2000):
• Simulations cannot replicate real-world complexity
• Simulations which approach real-world complexity would beexpensive to build
• Simulations can be misleading by focusing on the wrong issues
Much time is often spent getting robots to perform simple tasks such as grasping because of me-
chanical issues. A great deal of time can be wasted dealing with these issues instead of working on the
cognitive problem which the robot is intended to solve. A simulator can have particular features enabled
121
or disabled to allow irrelevant problems to be circumventedto allow the researcher to concentrate on
the ones of interest – an example of this is the grab functionality of the iCub simulator.
However it is always necessary to construct at least one robot, or a set of physical models represent-
ing different parts the robot, to confirm that the simulator is actually modelling the “real world”. For
example picking up an object requires an understanding of the physical characteristics of the surfaces
involved and this can only be determined experimentally.
The fidelity and validation of current simulators ranges widely. Towards the bottom end is Simbad
(Hugues & Bredeche, 2006) with no corresponding real robot,through EyeSim Koestler & Braunl
(2004) which adds simulated noise to the sensor inputs. At the top end is USARSim (Pepper et al.,
2007a) where great efforts have been made in validation, forexample in its the simulated Talon explosive
ordnance disposal robots (Pepper et al., 2007b).
A good case for a high fidelity mobile robot simulator has beenmade by Torrance (1992), although
both he and Rodney Brooks (Brooks, 1992) are adamant that a simulator must be based upon the real
characteristics of a real robot. Brooks has generally criticised simulators, noting that the differences
between simulator and real robot usually prevent an accurate understanding of the problems encountered
in the real robot, and pointing to the differences in actuation and sensing between simulators and the
real world.
Torrence makes the interesting comment that one criticism of simulators, that they require such a
good understanding of the real domain to simulate it, may actually make it is easier to understand how to
carry out the task in a real robot. He adds that trying to understand the environment in that much detail
might be useful in itself since this level of understanding can provide input to drive cognitive science
research.
E.1 Complexity of the simulator
High fidelity simulators are complex, and potentially expensive, to produce, but do not have to simulate
the entire world, only the environment that the robot inhabits – perhaps a well-lit table and a range of
objects. The effort to simulate these properly, and the computing power needed, should not be under-
estimated but once a suitable framework is in place then the simulated environment can be enlarged
without too much difficulty. Simulating the sensors also requires an accurate understanding of them, for
example to add the correct types of noise. The main problem might be video processing, for which a
considerable amount of computing power might be needed for accurate ray tracing to produce realistic
simulated camera inputs.
Such solutions in software can be used to explore different,randomised, scenarios in parallel. This
would allow stochastic learning to be accelerated, geneticprogramming techniques to be applied, and
Monte Carlo techniques to be used to model the robot’s behaviour under a range of conditions.
Many of the criticisms of simulators which have become the accepted view originated decades ago,
a time when the computers available were orders of magnitudeslower.
122
E.2 Embodied cognition
It may be important to use robots which cannot be physically constructed to study embodied cognition.
For example the iCub is intended to replicate a small child tohelp gain an understanding of human
embodied cognition, and yet despite being at the pinnacle oftechnical accomplishment it is very limited
compared in its sensors compared to its living counterpart.It would be possible to build a much richer
simulator, provided always that the components were validated against reality, even if a complete robot
is technically infeasible. For example a small area of real “skin” is all that is needed to to validate
covering over the entire simulated robot without having to address the data transfer problems of a large
number of sensors.
E.3 Conclusion
There is clearly a place for both high quality simulators andreal robots. Real robots are always needed,
and high quality simulators must always be validated properly against them. But the poor quality of
most current robotic simulators feeds the view that software simulation is a poor substitute for real
robots, which in turn discourages the development of bettersimulators.
123
Appendix F
Contents of the CD
The CD contains the following items.
• sourcecode.pdf, a document containing the source code written for this thesis.
• library , the library directory from the jyPOSH tree on sourceforge,with the sub-directories in it
trimmed to contain the two libraries created for this project – iCub, the cube handling task and
iCubTest, an initial experiment with YARP.
• simfork directory tree , from Sourceforge. This is a fork of the official iCub simulator that I have
made so that I could make some modifications, as documented inSection B.5.
• robot.avi, An AVI movie showing the robot performing the cube-handling task.
• log.txt, a log file of the run generating the movie showing the trace ofBOD behaviours being
triggered.
• refman.pdf, reference manual for the C++ control classes, generated byDoxygen.
• readme.txt, a text file describing the contents of the disk.
The runnable software is platform-dependent and depends upon more than 15 software packages
including YARP, gfortran and ACE. It typically takes 1-2 hours to install from a clean Ubuntu desktop
system. Instructions to do so are given in Appendix B. The jyPOSH and Simfork packages, which
include the code written for this thesis, can be copied from the CD.
124
