The Cortexionist architecture: behavioural intelligence of artificial creatures

Vis Comput (2010) 26: 353–366DOI 10.1007/s00371-010-0424-3

O R I G I NA L A RT I C L E

The Cortexionist architecture: behavioural intelligence of artificialcreatures

David Panzoli · Sara de Freitas · Yves Duthen ·Hervé Luga

Published online: 11 February 2010© Springer-Verlag 2010

Abstract Traditionally, producing intelligent behavioursfor artificial creatures involves modelling their cognitiveabilities. This approach raises two problems. On the onehand, defining manually the agent’s knowledge is a heavyand error-prone task that implies the intervention of the ani-mator. On the other hand, the relationship between cognitionand intelligence has not been theoretically nor experimen-tally proven so far. The ecological approaches provide a so-lution for these problems, by exploring the links betweenthe creature, its body and its environment. Using an artifi-cial life approach, we propose an original model of memorybased on the synthesis of several neuroscience theories. TheCortexionist controller integrates cortex-like structure intoa connectionist architecture in order to enhance the agent’sadaptation in a dynamic environment, ultimately leading tothe emergence of intelligent behaviour. Initial experimentspresented in this paper prove the validity of the model.

Keywords Computer animation · Autonomous adaptiveagents · Cognitive modelling · Human memory

D. Panzoli (�) · Y. Duthen · H. LugaIRIT-CNRS, UMR 5505, Université de Toulouse, Toulouse,Francee-mail: [email protected]

Y. Duthene-mail: [email protected]

H. Lugae-mail: [email protected]

S. de FreitasSerious Games Institute, Coventry University, Coventry, UKe-mail: [email protected]

1 Introduction

Realistic human-like agents are nowadays able to followgoals, plan actions, manipulate objects [1], show emo-tions [2] and even converse with humans. Despite theseagents being endowed with many abilities, the question ofintelligence, even for simple animal agents, is still beingconsidered. Indeed, intelligence is not necessarily related tothe ability to manipulate objects or synthesise speech, fol-low goals or plan actions. Many machines around us can dothat although they may not be considered intelligent.

From the earliest work of Alan Turing [3], artificial intel-ligence (AI) has considered that building an intelligent sys-tem implies the imitation of human mental processing, whatis referred to as cognitive modelling. In practice, imitatingthe way the human mind works involves writing by handcomplex algorithms, scripts, or sets of rule, the relevance ofwhich mostly depends on how they are interpreted.

Lately, radically different research works proposed newways to understand and consider intelligence. Pfeifer andBongard [4] introduced the concept of embodiment, whichforegrounds the major role that the environment plays in thecognitive abilities of any creature that lives in it. In paral-lel, Jeff Hawkins introduced the memory-prediction frame-work [5], a general theory of the neocortex, emphasising therole of memory in intelligence.

Our research is positioned within an artificial life (AL)context, and builds upon recent work of the team at IRIT [6].Lassabe and colleagues suggest that endowing a virtual crea-ture with realistic behaviours implies that this creature’smorphology emerges from the environment, taking away theneed of designing these behaviours by hand. Following asimilar idea, the work presented in this paper intends to showin addition that the behavioural intelligence of this creatureshould also be strongly connected to the complexity of the

mailto:[email protected]




354 D. Panzoli et al.

environment, as opposed to relying on human expert cogni-tive modelling.

Section 2 introduces the related work of the discipline,reviewing the trends in agent design in animation. Section 3sets out the contributions of the present work. Section 4 de-tails the foundation of our approach, notably through the in-vestigation of the relation between intelligence and the cor-tex. In Sect. 5, we present the Cortexionist model: a reactiveconnectionist architecture endowed with the ability to dealwith inner representations. This section emphasises the wayknowledge is created, stored, maintained and used through aprocess we call extended action-selection. Section 6 presentsexperimental data outlining how the model is expected to re-veal intelligent behaviour. Some interesting results are alsopresented and discussed. Section 7 proposes an extensionto increase the accuracy of the Cortexionist model. Finally,Sects. 8 and 9 provide a conclusion and a discussion aboutfuture work.

2 Related work

In the field of behavioural simulation, hybrid architectures[1, 7–10] have been the most competitive so far. These be-havioural controllers aim to combine reactiveness and cog-nitive abilities. A first reactive component provides a directinterplay between perception and action in order to ensurethe reactiveness of the action-selection mechanism [11]. Inparallel, a deliberative component is responsible for mod-elling the cognitive abilities, such as planning, reasoning orcommunicating. Basically, AI algorithms such as A* are ap-plied to world representations of the environment. Theserepresentations are organised through topological or gridmaps where the objects of the environment are themselvesrepresented symbolically such as Frames [12] or Smart Ob-jects [13].

Cognitive modelling makes clear that endowing au-tonomous virtual characters or creatures—hereafter referredto as agents—with a memory of knowledge representationsenhances their general behaviour, thus enabling them to in-teract with the environment, including with the other agents.However, the question of how this knowledge is designedraises two problems. First, the animator is expected to de-sign every feature of every object in the environment, notonly their properties and interacting possibilities (e.g. theaffordances), but also the relation they maintain with oneanother. Not only is this a tedious task, but it is error-proneas well. Secondly, a well-known weakness of symbolic rep-resentations is the symbol grounding problem [14]. Briefly,Harnad states that whatever one can expect with manipulat-ing symbols, “relating the symbols to the agent’s perceptionis coextensive with the problem of cognition itself”. This

reveals the lack of integration of the knowledge in the be-haviour, usually stressed by the separation of memory andcontrol into distinct modules in the controller’s architecture.

As a consequence, although cognitive modelling inher-its fifty years of AI expertise, it has begun to lose groundto the benefit of more ecological approaches [4, 5] whichplace an emphasis upon the predominant role of adaptation,thus considering cognition as a consequence rather than aprerequisite for intelligence. In this perspective, AL archi-tectures have regained popularity. Although they appear asmore limited than hybrid architectures, owing to their rela-tive simplicity, they give emphasis to the adaptation of thevirtual creature to the detriment of cognitive abilities, andoffer the opportunity to bring together memory and controlinto a simple and lightweight system.

3 Contribution

Traditionally, AL approaches focus on action-selection.Since action-selection consists of associating perceptionwith action, AL considers adaptation as the ability to detectwhether such an association is not relevant and hence per-forms changes in the controller in order to improve it. In thispaper, one original vision of adaptation is investigated. Webelieve the interest of a memory does not reside in the wayrepresentations are stored or can be manipulated but ratherin the representations themselves, particularly how they canenhance the adaptation of the agent to the environment.Moreover, we postulate that the ability to create patternsof knowledge from the environment, and take advantage ofthem to improve action-selection may reveal intelligent be-haviour, insofar as the environment allows it. Indeed, weassume that intelligent behaviour cannot be expected from acreature operating in a trivial environment.

The contribution of this work is dual. First, we provide amethodology for integrating a memory into a reactive agent,using an approach that is primarily concerned with avoidingthe weaknesses of hybrid architectures. This memory, then,must be integrated into the action-selection (e.g. does notrequire to be user-defined) and grounded upon perception(e.g. not symbolic). We then prove through experiments thatmemory is a bridge between complexity in the environmentand intelligence of the behaviour.

4 Foundations

The term Cortexionist, firstly introduced in [15], is a ne-ologism formed from cortex and connectionist. The latterpart derives from the connectionist foundation of the model.We believe indeed that neural networks provide an adequate

The Cortexionist architecture: behavioural intelligence of artificial creatures 355

Fig. 1 McLean’s hypothesis considers that the brain is made of threelayers, inherited from evolution

way to model the brain mechanisms, since they are a com-putational abstraction of the structures of the nervous sys-tem. However, simulating an entire brain is a more complextask. The last ambitious project [16] to date intends to sim-ulate the brain of a mouse. Using the IBM BlueGene super-calculator, the model could simulate 8 million neurons—which is half the actual size of a mouse brain—every oneof each having 6300 synapses, for a duration of 10 seconds,at 10 times less the actual speed of the brain. Although thissimulation was not a failure, it turned out to be unable to out-put any exploitable results. The reason invoked is the lack ofcurrent computing power, as compared to the high compu-tational complexity of the brain, as every neuron is linked tothousands of others.

Trying to reduce this complexity often means dividingthe model into sub-parts, each of them being responsible forthe simulation of a sub-system or an identified function inthe brain. The Psikharpax [17] project, for instance, aims tomodel the brain of a rat. It is composed of many modules,each responsible for a single task, such as navigation [18]or action-selection [19]. This ‘divide and conquer’ approachis successful but, as Gibbs notices: “scholars seem more in-terested in studying parts of people than they are in figuringout how the parts work together to create a whole, intelligenthuman being” [20].

As a matter of fact, we believe the reason why the previ-ous attempts to model the brain were unsuccessful is not apurely computational limitation, nor is it a misunderstandingof how the brain areas are connected together. We believe itis rather the consequence of our inability to understand themechanism of intelligence as a whole process, and to pro-vide the right structures to adequately model this process. Itis a fact indeed that evolution has favoured the appearance ofstructures in the brain, and it is likely as well that the appear-ance of new abilities is related to these new structures. Thisidea is the very centre of McLean’s triune brain theory [21].McLean hypotheses that the brain is actually composed ofthree components, which successively appeared during evo-lution (Fig. 1).

The reptilian brain, or archicortex, appears when the fishleft the water to populate the ground as batrachians. It isthe first central nervous system, totally insensitive to learn-ing and applying stereotypical and rigid schemes. It bringsthe first natural instincts (survival, aggression and territorialinstinct). Anatomically speaking, the reptilian brain corre-sponds to the brain stem and basal ganglia.

The limbic brain, or paleocortex, appears progressivelywith some mammals, bringing some specialised areas re-sponsible for emotions, fear or desire, but above all a moti-vation centre that introduces the notions of success and fail-ure. As a consequence, the limbic brain allows the creatureto learn, by associating situations with feelings, and thusfostering the adaptation of the creature to its environmentor a social group. The limbic brain regroups numerous sub-cortical structures, the most important being the hippocam-pus, the hypothalamus and the amygdala.

The neocortex is the most recent layer, known to be re-sponsible for imagination, abstract thinking and conscious-ness. It appeared with the big mammals, increasing in im-portance with the primates, and finally peaking with humanbeings. It is important to note that in the course of evolution,the neocortex has always expanded—from 410 cm2 in Aus-tralopithecus to 1400 cm2 with Homo Sapiens—and everygrowth phase has always reflected a significant evolution oftheir abilities.

Although McLean’s theory is still debated today, and sci-entists prefer to refer to brain structures (the basal ganglia)or systems (the limbic system) instead of layers, it still pro-vides an insight with how evolution has gradually ‘designed’structures to allow the creatures to learn, therefore leadingto the emergence of the individual over the species. Besides,studying the relations between the cortex and the rest of thebrain has the potential to elucidate the relation between be-haviour and intelligence.

Hawkins’ memory-prediction [5] framework1 provides asimple yet insightful view of how the cortex works. It hasbeen observed that the cortex was horizontally split into 6layers, C1 to C6 in Fig. 2, and vertically into many columns,called cortical columns. Studying how these columns werefunctioning, Hawkins noticed that, despite their apparentcomplexity, they were dedicated to a really simple process:associating and delaying neural signals in order to storetemporal patterns. These patterns exist at different levels ofcomplexity (a musical piece is perceived as a sequence ofmusical notes, each note being itself perceived as a sequenceof variations, etc.).

In parallel to storing these sequences, the cortex con-stantly makes predictions about the next perception, at all of

1The memory-prediction theoretical framework, issued to the Hierar-chical Temporal Memory model, is supported by an implementationbased on Bayesian networks [22].


Fig. 2 The thalamus plays a major role in the learning of sequence, bydelaying and sending back the data sent by every cortical column. Thisfigure schematises a cortical column

the different levels of complexity. Depending on this level,these predictions express the ability to recognise a knownobject from some of its features, or to anticipate a situationon the basis of a few clues. Hawkins’ opinion owes much tothe ecological approach. Basically he states that human in-telligence is based on the fact the environment is structured,and is catalysed by the ability to detect and represent thesestructures (in the cortex).

5 Description of the model

The Cortexionist model is founded on a simplified hypothe-sis. The human brain benefits from a long evolution, duringwhich several structures have appeared, enabling the humanbeings first to behave, then to learn from and adapt to theirenvironment, and finally to become intelligent. We think hu-man beings not only owe this last evolution to the appear-ance of the cortex, but to the way it interacts with the pre-vious structures. Therefore, the model aims to emulate andinvestigate this interaction by transposing a neocortex-likestructure (e.g. able to detect, capture and mirror the struc-tures in the environment) onto a computational limbic sys-tem (e.g. a structure responsible for adaptive behaviours).From the interaction between these two computational struc-tures, we hypothesise that we will be able to observe theemergence of behavioural intelligence.

5.1 Control

The model we propose here is based on a pure AL agent.The AL view is that the agent’s adaptivity is closely tied to

the controller’s ability to be modelled, through learning orevolution. Connectionist controllers have regularly provedsuitable for accurately shaping a desired behaviour [23–25].

Our controller is based on the simplest connectionist con-troller, a perceptron with two layers: one input layer for theperception and one output layer for the actions. The sensorsof the agent are bound to neurons from the input layer. Ina similar way, neurons from the output layer are bound tothe agent’s actuators. Feedforward neural connections linkthe input layer to the output layer. Finally, these connec-tions are tweaked during a training stage by a classic back-propagation algorithm [26].

Basically, the action-selection works as follows: a stim-ulus from the environment activates neurons on the inputlayer. The resulting signal is spread to the output layer whereoutput neurons are either activated or not, depending on theconnections. The activation of output neurons is translatedinto actions to be performed by the agent in the environment.At the end of the cycle, the agent faces a new situation andthe loop starts over again.

5.2 Knowledge representation

Beyond the reactive control of the agent’s behaviour, ourmodel gives emphasis to the ability for building inner repre-sentations.

The use of neural networks is totally appropriate consid-ering they are firstly a metaphor of human nervous struc-tures, such as the brain. From the connectionist point ofview, memory is distributed and highly associative [27],which means every piece of knowledge is built from theassembly of neurons operating throughout the brain, as as-sociative networks. These networks (see [28] for a survey)are dedicated to reproducing the mechanisms underlying thecreation, maintenance and destruction of knowledge. To thatend, they rely on associative rules, all of which are derivedfrom the original Hebbian rule [29].

In our model, for the purpose of seamlessly integratingthe memory inside the controller, such a rule applies to ex-isting neurons of the controller. More precisely, it applies tothe input layer as knowledge has to be grounded upon per-ception.

5.2.1 Creating knowledge

The unsupervised Hebbian rule states that co-activated neu-rons tend to strengthen their mutual connections, in such away that the mere exposure to co-occurring features leads totheir association into a piece of knowledge.

In our model, perceiving several features with one an-other in the environment results in the activation of their re-lated neurons on the input layer, thus leading to the creationof a pattern of knowledge as shown in Fig. 3.


Fig. 3 A pattern is created by applying an associative law. Let us con-sider a set of neurons (a). When a subset of neurons is co-activated (b),their mutual connections are strengthened in order to form a patternof knowledge (c). A black arrow on top of an input neuron means theneuron is activated by its related sensor

5.2.2 Retrieving and generalising knowledge

The main interest in creating patterns of knowledge is thepossibility of retrieving it subsequently from a few clues.This is achieved by a property inherent in associative net-works called pattern completion (described in Fig. 4).

Interestingly, this property is also the very foundation ofthe ability to transfer or generalise knowledge. Indeed, boththese processes rely on the ability to detect the proximity oftwo patterns of knowledge. As knowledge is distributed intonumerous neurons, the similarity of a pattern with anotherlogically relates to the number of neurons shared by bothpatterns. Knowing that, scaling the degree of generalisationin our memory can be attained by amending pattern comple-tion, and by adjusting the weight value of each participatingconnection when building a pattern.

Let us have a closer look at how this may work. We con-sider binary neurons, with a Heaviside transition functionwhose threshold s is set to 0.65. This means each neuron isactivated when the sum Σ of entering signals is higher than0.65. We fix arbitrarily the completion threshold Sc to 50%,so that half (or more) neurons of a pattern are necessaryto activate the whole. When n neurons are co-activated, theweight value of each connection of the new pattern is com-puted as follows:

w = s

n × Sc. (1)

Figure 5 shows how an entering signal may or may notbe associated with a known pattern of knowledge.

Finally, we need to make sure that overlapping knowl-edge does not prevent pattern completion from working

Fig. 4 Pattern completion is quite a simple rule: when a subset of neu-rons from a pattern is large enough, the whole pattern is activated. Thiscan be understood in this example by neurons A, B and E propagatingto C

Fig. 5 The weight value of each connection inside a pattern relieson the pattern size and the completion threshold. (a, b) In this exam-ple, applying the above formula during learning sets each weight tow = 0.65

3×0.5 = 0.43. In (c), neurons A and C are activated. The enteringsignal in neuron B is Σ = 2 × 0.43 = 0.86. Completion happens andB is thereby activated because 0.86 > 0.65. In (d), only neuron A isactivated. The entering signal in neurons B and C is 0.43, which is notsufficient compared to the activation threshold. Completion does notoccur

properly. Indeed, dealing with distributed representationsimplies that each pattern is likely to share one neuron ormore with others. We hypothesise that the rule can apply tooverlapping patterns without any modification, as shown inFig. 6.

5.2.3 Unlearning knowledge

Although learning new knowledge is the most importantability of a memory, unlearning is another fundamentalproperty in order to keep this connectionist memory fromsaturation and to accommodate changes in the environment.


Fig. 6 Two patterns Pi and Pj are likely to overlap when Pi does notshare enough neurons with Pj to cover it. (a, b) {A,B,C} cannot becompleted so that a new pattern {C,D,E} is formed aside. (c–e) Despitetwo patterns overlapping, the completion rule applies normally

Although this work does not cover a comprehensivestudy about forgetting in neural networks, it appears that theprevailing model considers that knowledge in memory de-cays in time, so that any material that is not often re-learntis likely to be forgotten eventually. We advocate a slightlydifferent idea. We envisage that forgetting something actu-ally means learning something different, although it is closeenough to overwrite the former knowledge. This section ex-plains how this may work in practice.

Once again, the process must be kept simple enough tobe seamlessly integrated into our associative network. Wepostulate that the simple use of inhibitory connections canhelp us solve the issue.

To date, we have used an associative rule to cohere neu-rons by building excitatory connections. The purpose of in-hibitory connections is, on the contrary, to dissociate neu-rons. Since co-activated neurons are associated, neurons thatare not activated at the same time should therefore be disso-ciated. Considering that only a few neurons (relatively to thetotal number of neurons) are co-activated every millisecondin the brain, it should be saturated with inhibitory connec-tions between non-coactivated neurons. The reason why it isnot is the existence of cerebral regions, which are brain ar-eas that regroup the neurons dealing with the same modalityand partition the modalities one from the other.

Fig. 7 Areas account for the creation of inhibitory connections. Whenseveral neurons belong to a common area (a), creating a pattern (b)involves the creation of excitatory connections between co-activatedneurons but also inhibitory connections with non-activated neurons ofthe area (c)

Transposed to the model, managing inhibitory connec-tions requires the associative layer to be partitioned into ar-eas that regroup neurons related to the same single sense.Inside such an area, we can presume that two neurons areunlikely to be activated at the same time, just as perceiv-ing an object is blue reliably involves it is not red. Figure 7shows how these patterns may include inhibitory connec-tions. Following from this, and given the ability to representthings that cannot co-occur, Fig. 8 shows how one patterncan replace another.

5.3 Integration and extended action-selection

Following the insight that inner representations of the worldcould help enhance the behaviour of the agent, we haveturned the input layer of a simple connectionist controllerinto a fully workable associative memory, imitating to acertain extent some of the most fundamental properties ofmemory.

Indeed, referring to the Squire and Cohen theory on pro-cedural and semantic memory [30] indicates we have ob-tained a complete model of memory. In the opposite di-rection of the traditional multi-store model of memory [31]that differentiates between short-term (STM) and long-termmemory (LTM), Squire and Cohen propose a more func-tional dichotomy between procedural memory, which bene-fits a supervised and iterative learning, and semantic mem-ory, whose learning is conversely unsupervised and non-iterative (e.g. ‘one-shot’).


Fig. 8 The ability to forget is illustrated by replacing a pattern withanother, close but still different. In (a), {A,B,C,E} represents an al-ready known pattern. C and D neurons inhibit each other since theybelong to the same area. When A, B , E and D are activated, D pre-vents the completion of the original pattern. Regardless, the associativerule applies and builds a new pattern {A,B,C,E}

Making a parallel with this theory, the initial perceptronis clearly an instantiation of a procedural memory, whereasthe associative network has all the features of a semanticmemory. From now on, we will use the term ‘procedural’ torefer to the perceptron connections, to the behavioural rulesthese connections are standing for, and to the learning ap-plied to the perceptron. We will however retain the term ‘as-sociative’ when referring to the associative network in orderto avoid any confusion, owing to the particular meaning ofthe term ‘semantic’ in computer science.

The model resulting from the integration of the associa-tive network into the perceptron, presented in Fig. 9, com-bines action-selection and sensory-driven knowledge repre-sentations. As a matter of fact, it also allows the investiga-tion of a deeper relation between procedural and semanticmemory.

Firstly, this new model works exactly like a standard per-ceptron, as the introduction of the associative network doesnot interfere with the action-selection (Fig. 10a and b). How-ever, the associative network’s ability to create and retrievepatterns of knowledge is responsible for some neurons onthe associative layer being activated by pattern completion,e.g. being activated, although they are not actually perceivedin the environment. As a result of these neurons also beingpart of the perceptron, as input neurons, they may lead to theselection of an action, if they happen to be part of a proce-dural rule (Fig. 10c).

In short, neurons inside the input layer can either be acti-vated by direct perception or by pattern completion. It means

Fig. 9 The behaviour of the agent, which can be a virtual characteror a robot equally, consists of performing actions in the environment.Actions are triggered by actuators, following the activation of outputneurons. Output neurons are themselves activated by input neurons,through the vertical propagation of the signal along the perceptron con-nections. Finally, input neurons can be directly activated by sensors,depending on the features perceived inside the environment, or indi-rectly by other input neurons, as patterns complete horizontally

Fig. 10 Standard action-selection (a) states that if there is a connection(e.g. a procedural rule) between the input neuron P0 and the outputneuron A0, the activation of P0 entails the activation of A0. Without aconnection (b), like between P1 and A0, this activation has no effect.The extended selection illustrates as follows (c): Considering now thecreation of the pattern {P0,P1}, the activation of P1 leads, owing topattern completion, to the activation of P0, which in turns activates A0

that beyond the procedural rules taught to the agent dur-ing the procedural training, new rules may implicitly appearwhen using knowledge formed during associative learning.We name this feature the extended action-selection. The nextsection presents our set of experiments in which we intend todemonstrate that intelligence may emerge from the extendedaction-selection.


Fig. 11 (a) The 3D simulation features a simple environment popu-lated with autonomous creatures. (b) The Cortexionist ‘aperçu’ pro-vides a visualisation of the controller

6 Experiments and results

Experiments are organised in a 3D virtual environmentwhere an agent (Agent) operates among other creatures:creatures of the same kind (Mother, Fellow) and predators(Predator). As shown in Fig. 11a, Agent is a small greencreature with a tentacle shape. Predator is a purple tentaclewith arms. Agent is also surrounded by other green grown-up tentacles, as Fellow. Mother is similar to Fellow but hasa particular smell that Agent is familiar with.

Every creature except Agent has a user-defined behav-iour. The aim of Predator is to catch and attack Agent, how-ever avoiding conflicts with other grown-up creatures, thatare able to defend themselves. In this way, Fellows andMother aim to protect Agent from Predator. In practice,Mother and Fellow are provided with a protection range. Ifthey can perceive both Agent and Predator within this range,they move towards Agent in order to offer some protection.

Agent is equipped with the Cortexionist controller, arepresentation of which is provided by a specific viewer(Fig. 11b). From several sensors, Agent’s perception istransported to dedicated neurons on the input layer: two forcolours (‘green’, ‘purple’), two for recognising basic shapes(‘tentacle’, ‘arms’) and another that detects the smell ofMother (‘smell’). Finally, Agent is also able to feel whetherit is colliding with another character (‘collision’). Such aperception is far from being based on a detailed imitation ofthe eye or the nose but still, it is non-symbolic since Agentis unable to directly recognise a creature. Agent’s action setconsists of two high-level actions: ‘move toward the closestcreature’ and ‘run away from the closest creature’.

Finally, Agent is provided with an initial amount of vi-tal energy. Logically, the energy expenditure depends on thespeed of Agent. When running away, Agent spends a lot ofenergy. When stopped or at low speed, Agent recovers someenergy.

The relevance of the model is proved through a compar-ison with a traditional connectionist controller. In practice,Agent is firstly confronted with situations without the abilityto create inner representations, which comes down to using

Fig. 12 Once adequately trained, the controller holds several con-nections between the perceptions and the actions. Red connectionsare excitatory, blue connections are inhibitory. In this case, thecontroller contains the following rules: {collision} → {run away},{smell} → {¬run away, move toward}

a purely reactive controller. Then, in the same conditions, itis given the ability to create inner representations and pos-sibly reveal a more intelligent behaviour. Regardless of thesituation, the controller initially contains some reactive be-haviours. These behaviours are taught to the agent by meansof a supervised training procedure, which aims to enforcethe following procedural rules.

The first rule makes Agent run away as soon as it col-lides with another character. This rule is derived from thetheory of the personal spaces of Hall [32]. The second rulelets Agent move towards Mother when it is smelled.

In practice, these rules appear as excitatory connectionsbetween perceptions and actions at the level of the controller(in red in the Fig. 12). Note that since these two rules arelearnt in parallel, the second rule implies the creation ofan inhibitory connection that prevents Agent from runningaway from Mother (in blue in the same figure).

Agent’s capacity to survive in the environment is mea-sured through two situations.

In the first one, Agent is introduced with Predator. Fig-ure 13 shows the significant steps of the simulation. Whenattacked by Predator, Agent runs away, then stops. Afterseveral attacks, the number of which depends on the initialamount of vital energy granted to Agent, it dies.

The second situation is more complex, as Mother andsome Fellows are introduced into the environment to helpAgent to escape from Predator. Figure 14 describes the mostsignificant steps of the simulation. In brief, Agent runs awaywhen it collides with Predator, but also with a Fellow. As aconsequence, in the absence of Mother and any kind of pro-tection, the Agent is condemned to die sooner or later underthe attacks of Predator.

Although Agent behaves in the exact way it has beentrained to, it is rather upsetting to see it unable to learn thatPredator is a threat even after being attacked several times.


Fig. 13 A purely reactive controller reveals an incapacity to learn thatthe predator is a threat. In (a) Agent is attacked by Predator. (b) Theimmediate action is to run away from Predator. (c) When Agent is at-tacked again, (d) it dies

The same conclusion is true in the context of cooperation asAgent is also unable to rely on a Fellow as long as it has notbeen explicitly taught to.

Besides, surviving these situations does not require elab-orate cognitive abilities but rather a better understanding ofthe environment. We may ask therefore what happens insimilar situations if the agent is able to create and make useof inner representations? This is what we are showing belowby introducing the Cortexionist agent in the same situations.

Creating representations is as simple as letting the asso-ciative rule apply while Agent faces different situations.

Figure 15 shows the Cortexionist agent in the first situa-tion and describes the significant steps of the simulation. Inbrief, Agent first builds a pattern of knowledge from the fea-tures it perceives from Predator. When it is attacked, Agentupdates the pattern to take Predator harmfulness into ac-count. Finally, Fig. 15c shows the resulting pattern in theassociative layer of the controller.

Figure 16 details the significant step of the Cortexion-ist agent in the second situation. For each step, the currentstate of the controller is provided. During the first 3 steps,Agent builds patterns of knowledge, related to Mother andthen to Predator. Note that the latter overlaps the first, asthe ‘tentacle’ neuron is common to both. However, the nextsteps demonstrate this has no incidence on Agent’s abilityto differentiate the creatures. Thanks to the Predator pattern,as demonstrated in the first scenario, Agent is able from this

Fig. 14 The second situation, featuring the reactive agent, can be sum-marised in 6 steps. (a) Agent is introduced in the environment in-side the protective range of its Mother. (b) As soon as Agent—slowerthan Mother—is distanced, it gets hunted by Predator. (c) Viewing thepredator has no consequence, as no rule in the behaviour applies in thissituation. (d) Predator colliding with Agent leads to the agent runningaway, while a nearby Fellow moves towards Agent to offer protection.(e) Fellow colliding with Agent triggers the same behaviour, leadingto Agent running away again. (f) Finally, Predator catches Agent onits way escaping Fellow, or any time later when Agent has run out ofenergy

point to run away from Predator from the mere sensing of itsmorphologic characteristics. When Agent perceives a Fel-low, the Mother-pattern is activated on the basis of the per-ceived characteristics. Recalling this pattern causes Agentto move towards Fellow, as it reminds it of Mother. Finally,colliding with Fellow has no more consequence, as the ‘runaway’ behaviour is innately inhibited. Besides, the ‘colli-sion’ neuron alone is also unable to retrieve the Predatorpattern.

Enabling the agent to use an associative rule on inputneurons has no perceptible effect on the procedural train-


Fig. 15 This figure illustrates the behaviour of the Cortexionistagent in the first situation. In (a) Agent faces Predator and a newpattern {‘tentacle’,‘arm’,‘purple’} is created, from assembling theco-activated neurons related to Predator’s features. In (b), as Agentis attacked, the previous pattern is updated into a new pattern {‘tenta-cle’,‘arm’,‘purple’,‘collision’}, taking into account this new situation.In (c), while Agent is running away, perceiving Predator, the activationof the ‘tentacle’, ‘arm’ and ‘purple’ features leads to the completionof the whole pattern, which includes the ‘collision’ neuron. In turn,the activation of this neuron triggers a ‘run away’ behaviour, keepingAgent away from Predator. (d) Shows the final pattern in the controllerof Agent

ing, so the reactive behaviours remain unchanged. Yet, us-ing these inner representations during the simulation in-escapably brings changes in the action-selection. Both casesdemonstrate how pattern completion helps retrieving knownsituations so that the relevant actions may be triggered. Tothe view of the observer, such a process may be manifestedas a form of anticipation (in the first situation) or generali-sation (in the second situation).

Different metrics have been investigated to assess theCortexionist model’s performance during the simulations.This section presents the most relevant metrics regarding theresult we want to stress.

The following curve (Fig. 17) compares the mean vitalenergies of both the reactive and the Cortexionist agents inthe first experiment. The data were gathered from severalsimulations where Predator’s speed was set to random val-ues around ±40% of a mean value. Statistically, the Cortex-ionist agent wastes less energy than the reactive agent, as itkeeps a safe distance to Predator.

The activity diagrams (Fig. 18) compare the behavioursof the reactive and the Cortexionist agents. They present the

Fig. 16 This figure shows the significant step of the Cortexionist agentin the second situation. For each step, the current state of the controlleris provided. (a) Agent is introduced in the range of Mother. Two oper-ations occur in the controller. First, the ‘smell’ neuron triggers a ‘movetowards’ behaviour. Then, the co-activation of ‘tentacle’, ‘smell’ and‘green’ are associated into a pattern. (b) Agent is now distanced, andfaces Predator. A new pattern {‘tentacle’,‘arm’,‘purple’} is created onthe associative layer, partly overlapping the previous one. (c) As Agentis now being attacked by Predator, the pattern related to Predator isupdated into a larger pattern that integrates the ‘collision’ sensing, justlike in the first situation (Fig. 15). In parallel, this collision triggers theaction to run away. (d) Agent, which is running away from Predator,anew perceives a Fellow. On the associative layer, pattern completionoccurs on the Mother-related pattern {‘tentacle’,‘smell’,‘green’} so thatperceiving the characteristics ‘tentacle’ and ‘green’, associated withthe Fellow, activates in turn ‘smell’. The activation of ‘smell’ entails inparallel a ‘move towards’ action, making Agent leading to the Fellow.(e) Agent collides with the Fellow, activating the ‘collision’ neuron.However, the ‘smell’ neuron prevents the activation of ‘run away’, sothat Agent stays under the protection of Fellow

effective activity, in terms of firing, of every neurons of thecontrollers, in such a way that neurons that participate theaction-selection are more easily identified. The activity dia-grams show very obviously that in the same situations (e.g.perceiving the same features) the two agents exhibit differ-ent behaviours. The behaviour has then been modified inter-nally and automatically, towards a better adaptation of Agentin its environment.

During the experimentations, some cases of failure oc-curred, revealing the model’s inability to cope with ambigu-ous perception. In the case presented in Fig. 19, as soon asintroduced in the environment, Agent both perceives Motherand Predator. As a result, all the features that belong toPredator and to Mother are regrouped into a single and use-less pattern in the controller (Fig. 19b), making Agent mis-takenly move towards Predator.

7 Discussion

The Cortexionist architecture has been presented as a sim-plified yet realistic model of the human memory. Firstly,procedural memory and semantic memory are represented,as well as their respective ways of learning. Then, and aboveall, this architecture differs from previous attempts to model


Fig. 16 (Continued)

the human memory by not following the traditional multi-store design [31, 33], to the benefit of a more functionalapproach. Indeed, although the multi-store models are stillvery popular when designing computational models—forthe convenient compatibility between the STM/LTM struc-ture and the way information is processed by a computer—Craik and Lockhart argued the lack of neuro-scientific foun-dations of this theory, making any discussion about the stor-age of information in the brain futile.

In their ‘levels of processing’ theory [34], they ratherinvestigate the way stimuli are processed by the brain.Through experimentations, they find out that a collectionof random letters is quickly forgotten whereas it can last awhole week if they form a word. Their conclusion is thatthe more processing a memory item involves, the longer itis likely to be held in memory. Furthermore, the stimulusseems to acquire more semantic meaning according to thetime granted to this processing. These results have led Craikand Lockhart to represent human memory as a multi-layeredstructure, where the deepest layers might present the great-est complexity or abstraction and the best durability.

We believe that taking this theory into account consti-tutes the next step towards making Cortexionist a more re-alistic model of human memory. Early studies have led tothe creation of Cortexionist 2.0, which adds more layers, ofincreasing complexity, in order to produce more complexbehaviours (Fig. 20). Basically, all the layers are workingon the same principle as the current associative layer, butthey rely on one another to recursively build more com-plex patterns from the association of patterns from the lowerlayer. The ‘co-activation frame’ is also stretched followingthe depth of a layer in such a way that more complex patternscan be created from lower complexity patterns that were notdirectly activated at the exact same time, but in a relativelyclose time-frame. This way, temporal pattern can be createdand then used to anticipate previously learnt situations.

8 Conclusion

To conclude, we have proposed in this paper an original con-troller for autonomous creatures where a simplified, how-ever realistic model of memory is integrated into an adap-tive reactive architecture. The main novelty of this worklies in our approach which is radically different from tra-ditional computational architectures. First, the knowledgerepresentations are grounded on the agent’s very percep-tion, defending the idea that the power of representation isthe representation itself, not the way it can be manipulated.Besides, these representations do not require any additionalwork from the animator. Then, this non-symbolic memory isseamlessly integrated into the controller, so that the knowl-edge directly participates in the action-selection. Complexbehaviours are then reflected by the creature’s ability to takeadvantage of a better understanding of the environment—namely, the structures and the rules that govern it—insteadof relying on the animator’s skills in writing scripts.

Experiments comparing the behaviour of the agent withand without the ability to create knowledge demonstratehow a better understanding of the environment entails a bet-ter adaptation, and therefore intelligent behaviours. The con-dition is then the complexity of the environment, which is


Fig. 17 This curve shows thatCortexionist agent wastessignificantly less energy than thereactive agent, owing to itsability to anticipate the attacksfrom the predator

Fig. 18 Activity diagramshighlight the fact that differentbehaviours may be selected onthe basis of similar perceptions.The difference between thereactive controller and theCortexionist controller isparticularly obvious from theblack arrow

Fig. 19 In this case, the Cortexionist model does not work as it is un-able to deal with ambiguous perception. As Agent is facing both Preda-tor and Mother (a), a useless pattern is created in the controller (b)

guaranteed in our study by introducing several autonomousand dynamical creatures.

9 Future work

Considering the simple nature of the experiments presentedin the paper, future work may be focused upon testing thecontroller’s ability to be scaled up to larger environments, byusing more creatures, a more complex environment, and inaddition by introducing ambiguous creatures—for examplea creature that looks harmless but is actually not.

The experiments consisted in comparing the Cortexionistnetwork to a very simple perceptron. However, more recent


Fig. 20 The Cortexionist 2.0 model is a prototype where several layersare stacked in order to allow the creation of knowledge of increasingcomplexity. The complexity is also tied to a time dimension, so that theextended action-selection also takes on a temporal aspect, allowing thecontroller to build temporal patterns an then anticipate already knownsituations

neural networks, such as recurrent neural networks [35] orNEAT [36], introduce recurrent connections in order to en-dow action-selection with a notion of context. Future workplans more comparisons with these networks, in order toprove the greater effectiveness of an associative layer overrecurrent connections.

Acknowledgements All videos from which the pictures of this pa-per are extracted can be found at the following URL: http://www.irit.fr/~David.Panzoli/

References

1. Sanchez, S., Balet, O., Luga, H., Duthen, Y.: Autonomous virtualactors. In: 2nd International Conference on Technologies for Inter-active Digital Storytelling and Entertainment. LNCS, Darmstadt,Germany, June 2004, pp. 68–78. Springer, Berlin (2004)

2. Pelachaud, C.: Multimodal expressive embodied conversationalagents. In: MULTIMEDIA ’05: Proceedings of the 13th AnnualACM International Conference on Multimedia, New York, NY,USA, 2005, pp. 683–689. ACM, New York (2005)

3. Turing, A.M.: Computing machinery and intelligence. Mind 59,433–460 (1950)

4. Pfeifer, R., Bongard, J.C.: How the Body Shapes the Way WeThink: A New View of Intelligence. The MIT Press, Cambridge(2006)

5. Hawkins, J., Blakeslee, S.: On Intelligence. Times Books, NewYork (2004)

6. Lassabe, N., Luga, H., Duthen, Y.: A new step for evolving crea-tures. In: IEEE-ALife’07, Honolulu, Hawaii, pp. 243–251. IEEE,New York (2007)

7. Shao, W., Terzopoulos, D.: Autonomous pedestrians. Graph. Mod-els 69(5–6), 246–274 (2007)

8. Conde, T., Thalmann, D.: An integrated perception for au-tonomous virtual agents: active and predictive perception: re-search articles. Comput. Animat. Virtual Worlds 17(3–4), 457–468 (2006)

9. Funge, J., Tu, X., Terzopoulos, D.: Cognitive modeling: knowl-edge, reasoning and planning for intelligent characters. In: The26th Annual Conference on Computer Graphics and InteractiveTechniques, pp. 29–38. ACM Press/Addison-Wesley, New York(1999)

10. Donikian, S.: HPTS: a behaviour modelling language for au-tonomous agents. In: Proceedings of the Fifth International Con-ference on Autonomous Agents, Montreal, Canada, 2001

11. Brooks, R.A.: A robust layered control system for a mobile robot.IEEE J. Robot. Autom., 14–23 (1986)

12. Minsky, M.: A framework for representing knowledge. Psychol.Comput. Vision, 211–277 (1975)

13. Kallmann, M., Thalmann, D.: A behavioral interface to simulateagent–object interactions in real-time. In: Proceedings of Com-puter Animation 99, Geneva, 1999, pp. 138–146. IEEE ComputerSociety Press, Los Alamitos (1999)

14. Harnad, S.: The symbol grounding problem. Physica 42, 335–346(1990)

15. Panzoli, D., Luga, H., Duthen, Y.: Beyond reactive systems: An as-sociative memory for sensory-driven intelligent behavior. In: Pro-ceedings of the International Conference on CyberWorlds, Brad-ford, UK, September 2009, pp. 63–70

16. Frye, J., Ananthanarayanan, R., Modha, D.S.: Towards real-time,mouse-scale cortical simulations. In: CoSyNe: Computational andSystems Neuroscience, 2007

17. Meyer, J.-A., Guillot, A., Pirim, P., Berthoz, A.: Psikharpax:An autonomous and adaptive artificial rat. In: Proceedings ofISR2004, the 35th International Symposium on Robotics, Paris,2004

18. Filliat, D., Meyer, J.-A.: Active perception and map learning forrobot navigation. In: From Animals to Animats 6, Proceedings ofthe Sixth International Conference on Simulation of Adaptive Be-havior, pp. 246–255. The MIT Press, Cambridge (2000)

19. Girard, B., Cuzin, V., Guillot, A., Gurney, K., Prescott, T.J.:A basal ganglia inspired model of action selection evaluated ina robotic survival task. J. Integr. Neurosci. 2(22), 179–200 (2003)

20. Gibbs, R.W.: A review of ‘how the body shapes the way we think:a new view of intelligence’ from Rolf Pfeifer and Josh Bongard.Pragmat. Cognit. 15(3), 610–614 (2007)

21. McLean, Paul: The Triune Brain in Evolution. Springer, Berlin(1990)

22. George, D., Hawkins, J.: A hierarchical Bayesian model of invari-ant pattern recognition in the visual cortex. In: Proceedings of theInternational Joint Conference on Neural Networks (IJCNN’05),2005

23. Cliff, D., Miller, G.F.: Co-evolution of pursuit and evasion II: sim-ulation methods and results. In: From Animals to Animats 4: Pro-ceedings of the Fourth International Conference on Simulationof Adaptive Behavior, pp. 506–514. The MIT Press, Cambridge(1996)

24. Terzopoulos, D., Tu, X.: Artificial fishes: physics, locomotion,perception, behavior. In: SIGGRAPH’94 Computer Graphics,ACM SIGGRAPH, pp. 42–48. ACM Press, New York (1994)

25. Blumberg, B.M., Todd, P.M., Maes, P.: No bad dogs: ethologicallessons for learning in Hamsterdam. In: Proceedings of the 4thInternational Conference on the Simulation of Adaptive Behavior,1996

26. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning represen-tations by back-propagating errors. Nature 323, 533–536 (1986)

27. Anderson, J.R.: The Architecture of Cognition. Harvard Univer-sity Press, Cambridge (1983)

http://www.irit.fr/~David.Panzoli/

http://www.irit.fr/~David.Panzoli/


28. Arbib, M.A. (ed.): The Handbook of Brain Theory and NeuralNetworks. The MIT Press, Cambridge (2002)

29. Hebb, D.: The Organisation of Behavior. Wiley, New York (1949)30. Cohen, N.J., Squire, L.R.: Preserved learning and retention of pat-

tern analysing skill in amnesia: dissociation of knowing how andknowing that. Science 210, 207–209 (1980)

31. Atkinson, R.C., Shiffrin, R.M.: Human memory: a proposed sys-tem and its control processes. In: Spence, K.W. (ed.) The Psy-chology of Learning and Motivation: Advances in Research andTheory, vol. 2, pp. 89–195. Academic Press, New York (1968)

32. Hall, E.T.: The Hidden Dimension. Anchor Books, Peterborough(1966)

33. Baddeley, A., Hitch, G.: Working memory. In: Recent Advancesin Learning and Motivation, vol. 8. Academic Press, New-York(1974)

34. Craik, F.I.M., Lockhart, R.S.: Levels of processing: a frameworkfor memory research. J. Verbal Learn. Verbal Behav. 11, 671–684(1972)

35. Hochreiter, S., Schmidhuber, J.: Long short-term memory. NeuralComput. 9, 1735–1780 (1997)

36. Stanley, K.O., Miikkulainen, R.: Evolving neural networksthrough augmenting topologies. Evol. Comput. 10(2), 99–127(2002)

David Panzoli is a Research As-sistant at the Serious Games Insti-tute of Coventry University. He re-ceived his Master‘s and Ph.D. inComputer Science from the Uni-versity of Toulouse, respectively in2003 and 2008. His research inter-ests include animation techniques,neural networks, artificial life andintelligent agents. He focuses on thedesign of lightweight systems forthe adaptive control of virtual char-acters evolving in dynamic environ-ments.

Sara de Freitas is Professor of Vir-tual Environments and Director ofResearch at the Serious Games In-stitute (SGI)—an international hubof excellence in the area of games,virtual worlds and interactive digi-tal media for serious purposes. Sit-uated on the Technology Park atthe University of Coventry, Saraleads an interdisciplinary and cross-university applied research groupwith expertise in AI and games,visualization, mixed reality, aug-mented reality and location-awaretechnologies. The research group

works closely with international industrial and academic research anddevelopment partners. Sara is a Visiting Fellow of the London Knowl-edge Lab, London, and a Fellow of the Royal Society of Arts.

Yves Duthen is a Research Pro-fessor of Artificial Life and VirtualReality at IRIT Lab, University ofToulouse 1, Capitole. He receivedhis Ph.D. degree from the Univer-sity Paul Sabatier in 1983 and the“French Habilitation” Postdoctoraldegree in 1993 to become full Pro-fessor. He has worked in image syn-thesis during the 1980s and focusedon behavioural simulation basedon evolutionary mechanism since1990. He has pioneered research inartificial life for building adaptiveartificial creatures and focuses nowon embedded metabolism.

Hervé Luga is Assistant Professorat Toulouse University. He receivedhis Ph.D. in Computer Science fromthe University of Toulouse III, PaulSabatier in 1993. His research con-cerns the automatic generation ofshapes, morphologies and behav-iours for virtual entities, relying onthe artificial life paradigm. He hassupervised several Ph.D. studentson the topic of generating complexbehaviours able to cope with inter-actions at both the reactive and thecognitive levels.

The Cortexionist architecture: behavioural intelligence of artificial creatures

Documents

Transcript of The Cortexionist architecture: behavioural intelligence of artificial creatures