Chapter 1Per eption-A tion Learning as anEpistemologi ally-Consistent Model forSelf-Updating Cognitive RepresentationDavid Windridge, Josef KittlerAbstra t As well as having the ability to formulate models of the world apableof experimental falsi� ation, it is evident that human ognitive apability embra essome degree of representational plasti ity, having the s ope (at least in infan y) tomodify the primitives in terms of whi h the world is delineated. We hen e employthe term ' ognitive bootstrapping' to refer to the autonomous updating of an embod-ied agent's per eptual framework in response to the per eived requirements of theenvironment in su h a way as to retain the ability to re�ne the environment modelin a onsistent fashion a ross per eptual hanges.We will thus argue that the on ept of ognitive bootstrapping is epistemi ally ill-founded unless there exists an a priori per ept/motor interrelation apable of main-taining an empiri al distin tion between the various possibilities of per eptual ate-gorization and the inherent un ertainties of environment modeling.As an instantiation of this idea, we shall spe ify a very general, logi ally-indu tivemodel of per eption-a tion learning apable of ompa t re-parameterization of theper ept spa e. In onsequen e of the a priori per ept/a tion oupling, the novelper eptual state transitions so generated always exist in bije tive orrelation with aset of novel a tion states, giving rise to the required empiri al validation riterion forper eptual inferen es. Environmental des ription is orrespondingly a omplishedin terms of progressively higher-level affordan e onje tureswhi h are likewise val-idated by exploratory a tion.Appli ation of this me hanismwithin simulated per eption-a tion environments in-di ates that, as well as signi� antly redu ing the size and spe i� ity of the a prioriper eptual parameter-spa e, the method an signi� antly redu e the number of iter-ations required for a urate onvergen e of the world-model. It does so by virtue ofthe a tive learning hara teristi s impli it in the notion of ognitive bootstrapping.David WindridgeCentre for Vision, Spee h and Signal Pro essing,Fa ulty of Engineering & Physi al S ien es,University of Surrey, Guildford, UKe-mail: d.windridge�surrey.a .uk 1

2 David Windridge, Josef Kittler1.1 Introdu tionAn autonomous ognitive agent is one apable of fun tioning within an unstru -tured environment. Typi ally, this fun tionality takes the form of the updating ofa ognitive agent's internal world-model in response to exploratory �ndings, andthe subsequent spe i� ation of appropriate goals within this revised domain [35℄.In the most general ases, an autonomous ognitive ar hite ture has the ability gainnew goal-setting apabilities on the basis of generalizations about its previous ex-ploratory �ndings (eg [19, 44℄). In prin iple, this ability to gain new apabilitiesshould extend to the possibility of autonomously revising the per eptual basis onwhi h the agent's world experien e are predi ated (that is to say, the manner inwhi h the world is represented).It is evident that (within ertain important onstraints) human ognitive agen ymust exhibit this apability. For instan e, a human adult may, within low- apa ityshort-term memory, hara terize an external s ene (its world model) as onsistingof the primitives ar, buildings, trees, et , in various states of spatial interrelation-ship. However, sin e none of these per eptual ategories are impli it in a newborninfant, the adult human must have a quired the per eptual ategorizations in the ourse of their life. To some extent, these per eptual ategories are so ial a quisi-tions, learned via intera tion with other humans (perhaps in the manner of [48℄).However, their initial delineation would still require explanation. More subtly (andat an epistemologi ally-deeper level), we might wonder why we, for instan e, ate-gorize the individually per eivable omponents hands, arms, legs, et as the singularper ept person. Equally, we might wonder why we do not naturally amalgamate thesu h sub-per epts as tree-bran h and house-window into a singular per ept 'bran h-window' ( f Quine's notion of 'ontologi al relativity' [37℄). It is not obviously the ase that these ategory-amalgamations are based on the innate per eptual (or phys-i al) distan e between sub- ategories: even if su h were the ase, it would not be lear that the parameters governing per eptual lustering and sub- lustering ouldbe innately spe i�ed so as to onstitute 'useful' ategories su h as 'person'.For a stati world, it is hen e evident that if an individual's ability to reate per- eptual ategories is not onstrained in some manner (so that we were, for instan e,free to amalgamate the sub-per epts tree-bran h and house-window into a single ategory), there ould be no absolute distin tion made between world-model andper eptual ategory. However, this distin tion onstitutes the fundamental basis of ognition as being the a t of 'knowing the external world'.Given this problem with stati distin tions, it is evident, then, that the apa ityfor the world to undergo hange is ru ial to its per eptual hara terization. Hen e,it is logi al to form the omposite per eptual ategory ar from its individual om-ponents door, wheels, winds reen, et , be ause these onstitute o-moving enti-ties. We thus impli itly onstitute at least some of our per eptual ategorizations onthe basis of per eptual o-arti ulation (that is, the mutual dependen y existing be-tween low-level translatable per epts). Independently-mobile entities (eg ars andhumans) thus onstitute separate per eptual ategorizations (even during momentsof intera tion).

1 Per eption-A tion Learning as Model for Self-Updating Cognition 3However, this only a ounts for the self-motive aspe ts of the world (those en-tities apable of initiating o-movement of individual parts without any externalinput). How then are we to a ount for the distin t segmentation of stati , ba k-ground per eptual entities su h as tree and building? To arrive at a meaningful basisfor these per eptual ategorizations, we need to onsider the motive possibilities ofthe agent; spe i� ally an a tive agent embodied within the s ene.The problem addressed in this paper is therefore that of unsupervized per eptual ategory reation in embodied agents. A entral on ern is thus how to render thispro ess empiri ally meaningful and free of foundational paradoxes (in the senseof the distin tion between world-model and per eptual ategory being lost). Wepropose the strategy of ognitive bootstrapping as a means to a omplishing this.We have elsewhere [2, 47℄ de�ned ognitive bootstrapping among autonomoussensory agents as the a t of spontaneously inferring new per eptual ategories in amanner that simultaneously allows for the ontinuous re�nement of models of theembodied agent's external environment des ribed in terms of those ategories. It isevident from the pre eding dis ussion that su h per eptual inferen emust be appro-priately onstrained in order to ensure that novel per epts are meaningful and non-arbitrary, and further that per eptual-frameworkre�nement and world-model re�ne-ment are kept distin t. That is, (for some hypotheti al [potentially surje tive℄ fun -tion, p :W! 2jPj, apable of relating a parti ularworld-datumW to a orrespondingset of observed per epts, fPg), we require that errors of representation, (eg of theform pobserver(fWpartialg) 6= poptimal(fWpartialg)), an be readily distinguished fromerrors of world-modeling (of the form pobserver(fWgeneralizedg) 6= pobserver(fWallg)).(The fun tion pobserver(fWgeneralizedg) is thus a generalized world model in the ob-server's 'frame' of per eptual referen e onstru ted from the partial observationspobserver(fWpartialg); the set fWallg is onsequently the totality of (potentially) mea-surable world data, and poptimal is the frame of per eptual referen e when optimizedvia some appropriate riterion).The hief epistemologi al danger for a self-updating ognitive system is thus thatany ompletely un onstrained apa ity for per eptual updating (for instan e, per-mitting pobserver to range freely over the entire fun tional spa e: p :W ! 2jPj) wouldhave the potential to a ommodate any given error- on�guration of the world-model (ie so that pobserver(fWgeneralizedg) = poptimal(fWallg) for pobserver 6= poptimaland fWgeneralizedg 6= fWallg ), and thus la k the ability to found the distin tion be-tween the obje tive world and its representation. An appropriate analogy for p hereis an orthonormal basis of per eptual primitives embedded within a Hilbert spa e ofworld-data ve tors (su h that the orthonormal per eptual primitives represent inde-pendent attribute lasses su h as olor and shape). Thus, while it is possible for anygiven set of world data and generalized world models to be represented in any givenbasis (in luding a orre t world-model within an in orre t basis), there always re-mains an optimal basis in whi h the distribution of world ve tors is most ompa tlyrepresented (in this ase when the marginal distributions of world-ve tors are max-imally non-Gaussian). Consequently, independent optimization of p andWgeneralizedis possible (by minimum des ription-length en oding and empiri al re�nement, re-spe tively) if there exists a per ept-independent way of generating the distribution

4 David Windridge, Josef Kittlerof world-data ve tors. It is the argument of the urrent paper that a tions pre iselyful�ll this riteria.A signi� ant aspe t of the survey endeavor of [46℄ was therefore to iden-tify a me hanism for empiri ally-meaningful ognitive bootstrapping re on ilableto the on erns of both ognitive s ien e (eg [6, 13, 43℄) and philosophy (eg[18,22,24,29℄). The identi�ed me hanism onsists in an a priori per eption-a tion oupling apable of progressive, open-ended abstra tion, su h that an embodiedagent equipped with it is able to form higher-level symboli generalizations that arealways grounded via orresponding a tions hierar hi ally onne ted to the a pri-ori per eption-a tion oupling. It is thus the aim of an agent employing ognitive-bootstrapping to maximize the general des riptivity of per epts in so far as they arerelevant to the agent's a tions.Hen e, any formalization of this notion will require per eptual inferen e to bedriven by the attempt to form a generalized mapping between per eptual statesand environmentally-legitimate a tions ommen ing from an initial, generi sensor-motor omplex (whi h, being generi , is not optimized for the spe i� environ-ment in whi h it is pla ed). This generalization an be a omplished by a varietyof ma hine-learning methods (an early approa h to symboli generalization of theper eption-a tion ouplingwas demonstrated via sto hasti lustering in [2℄, thoughwithout the system having the generi ity required to suggest a tions existing out-side and agent's previous experien e). The on ern of the urrent paper is hen eto present a method for a hieving this generalization in the most universal logi alterms, in su h a way as to enable entirely novel a tion lasses to be indu ed by theagent.1.1.1 Cognitive Bootstrapping in a Per eption-A tion ContextThe importan e of a per eption-a tion (P-A) learning framework to ognitive boot-strapping arises from its reversal of the lassi al approa h to agent-based learn-ing, in whi h per eptual inferen e typi ally pre edes exploration. (Su h an agentforms a per eptual model of the world prior to attempting to intera t with it).Instead, the per eption-a tion agent performs per eptual inferen e only after ex-ploratory a tions have taken pla e [10, 14, 15, 40℄. Hen e, the P-A agent's worldmodel is onstru ted only from those per epts that hange as the result of theagent's a tions. Thus for a set of a priori per epts fPg, and a set of a tions fAg,the agent's world model is some generalization, fgen, of the partial set of mappingsf : fAexpg! fPexpg�fPexpg brought about by the set of exploratory a tivities Aexpapparent in the per ept spa e fPexpg (where fPexpg 2 fPg). (This mapping has theform of a Cartesian produ t in onsequen e of the fa t that an individual a tion Alinks a per ept Pinitial [existing at the outset of the a tion℄ to a per eptPf inal [existingat the end of the a tion℄). Note here that we have assumed for simpli ity (in distin -tion to the more general Hilbert-spa e example given earlier) that the set of a prioriper epts fPg is su h that world on�gurations have an unambiguous representation

1 Per eption-A tion Learning as Model for Self-Updating Cognition 5as a spe i� singular per ept; that is, we assume that per eptual attribute lasses thatare fully orrelated with ea h other with respe t to a tions exist in some unspe i�edper eptual sub domain so as to keep the fo us here purely on the per ept-a tion re-lation. The generalized world-model fgen hen e represents the P-A agent's beliefsabout its potential to undertake a tions within the world having dire tly observ-able onsequen es. The a priori per epts fPg are thus very general; they might,for instan e, represent all of the distinguishable on�gurations of a visual sensor,su h as a amera. (Hen e, for an n-state, X �Y -sized mono hrome amera array,jfPgj= nX�Y ). However, while it is ne essary that fgen has a domain and range de-rived from the sets fAg and fPg�fPg, respe tively, it is unlikely that the mappingwill prove exhaustive: the a tions available to the agent, fgen(Agen), will, in general,only be able to a ess a subset of the total range of per eivable states fPg.It is hen e very unlikely that the a priori per ept set fPg is the most ef� ientmeans of representing the world. If the non-redundant (ie intentionally a essible)set of per epts brought about by exploratory a tivity fAexpg is denoted fP0expg, thenwe would expe t that the generalized fun tion fgen has an equally generalized rangefP0geng deriving from fP0expg1. The set of generalized per epts fP0geng hen e onsistsin an abstra ted, higher-level set of per eptual ategorizations that are predi atedpurely on the basis of the agent's a tive apability.To see how this might apply in pra ti e, onsider a pra ti al example in whi h ana priori per eptual des ription is given of a dis retized Cartesian spa e, X , su h thatea h lo ation an have a parti ular label sele ted from a set, L. This orrespondsto the typi al assumptions underlying standard image pro essing systems; ie anintensity-labeled pixel-grid. Under a per eption-a tion bije tivity onstraint of theform, fPg�fPg, fAg (see se tion 1.1.2), the set of all on eivable a tions orre-sponds to the set of possible per eptual transitions in the a priori spa e; ie the bije -tivity has the form: fjLjjXjg�fjLjjXjg,fAg. Now suppose that an embodied ogni-tive agent equipped with this visual system determines via randomized exploratorymotor a tivity (that a tivates, say, a series of robot limbs) that one of these labelspersists over the exploratory transitions, ie that l ! Xinitial & l ! X�nal for some(l 2 L), ( Xinitial;X�nal 2 X). A natural generalization of this assumption is that all la-bels, l, persist for all transitions, ie 8(l;Xinitial;X�nal) : l!Xinitial & l!X�nal (l 2 L),where Xinitial;X�nal 2 X .However, under this generalized assumption, the spa e of possible transitionsis vastly redu ed, being now of magnitude fjLj� jX jg�fjLj� jX jg, sin e ea h ofthe jLj obje ts an only o upy one of jX j lo ations. Under the per eption-a tionbije tivity onstraint (fP0g�fP0g, fAgeneralizedg) governing revisions fP0g of theper ept spa e fPg in the light of experimental data that indi ate the existen e of ageneralized onstraint, fAgeng, upon the a tion-spa e fAg ( Agen � A), we now haveinstead that: fjLj� jX jg�fjLj� jX jg, fAg.That is to say, the new per ept-spa e, fP0g= fjLj� jX jg, has the form of a poly-nomial of �rst order in jX j, while the former per ept-spa e, fP0g= fjLjjXjg, is expo-nential in jX j. Given that jX j is typi ally large, this is a very signi� ant e onomiza-1 fP0expg is a subset of fPexpg sin e, in general, not all of the exploratory a tions will result inper epts that ould be regarded as intentional; for instan e, if an a tion end-point is unstable.

6 David Windridge, Josef Kittlertion of the per ept domain. Furthermore, this alternative per ept domain is a subsetof the original domain (sin e the a priori spa e, fPg, must be hosen so as to over-populate the spa e of a tion possibilities, in order that environmental onstraints an be determined at all).In general, following any su h per eptual updating, we would still wish to retainthe original a priori per ept spa e so that any per eptual transitions not a ommo-dated by the new per eptual spa e ould still be per eived, enabling the proposedper eptual transition to be reversed if unsuitable. We thus make expli it the im-pli it subsetting P0 � P and onsider, instead of omplete transitions from one per- eptual domain to another, rather the hierar hi al extension of the per ept spa e:P0 � P0 � P00 � P000 � : : :. Hen e, exploratory a tivity takes pla es at the apex ofa re ursively-inferred per eptual hierar hy, rather than within the original a priorispa e; lower levels of the per eptual hierar hy thus a t to progressively ontextu-alize the in reasingly abstra t a tion-spe i� ation o urring at higher levels of thehierar hy. In this way, (ie via re ursive hierar hi al abstra tion of the per ept spa e),we automati ally onstru t a grounded symbol-generating and manipulating agent.Criti ally, in as ending the per eptual hierar hy, we still retain the ability to fal-sify spe i� a tion transitions, a 2 fA0g, and thus retain the ability to onstru t afalsi�able model of the world in a ordan e with the standard (ie non-per eptuallyupdating) on eption of a learning agent.Autonomous, exploration-based abstra tion of an a priori per eption-a tion ou-pling hen e provides a means of updating both an agent's per eptual framework aswell its model of the external world, without in urring the foundational paradoxeswe might expe t in attempting to employ a per eptual framework to postulate theexisten e external of obje ts, and those same obje ts to postulate the existen e of al-ternative per eptual frameworks. Hen e, by onstraining proposed per eptual mod-i� ations to be hierar hi al abstra tions of the a priori per eptual framework fPg(whi h annot themselves be subje t to empiri al on�rmation or refutation), we al-ways ensure that both the per eptual framework and the obje ts per eived in termsof it (ie fgen) are subje t to empiri al veri�ability.A further advantage of this strategy is that issues of framing [8,27℄ are largely re-solved, sin e the agent eliminates mu h of the operationally-redundant environmentdes riptivity that o urs in lassi al approa hes to autonomous ognition.In an agent that employs per eption-a tion learning a tively2 we might thereforeexpe t that the generalized per eptual hypotheses built on exploratory a tions ouldthemselves serve to guide the exploration pro ess. They would do this by suggest-ing novel per ept states to whi h hypothesized agent a tions ould transit (ie theset of a tions fA0geng that onne t the novel per epts fP0geng together via fgen ). Ex-ploratory a tions in this s enario would hen e serve to re�ne both obje t and per epthypotheses at the same time, bootstrapping obje t and per ept models in responseto environmental demands.It is hen e the endeavor of this paper to exploit per eption-a tion theory to setout a ognitive bootstrapping me hanism for autonomous logi al agents apable2 Cf eg [3, 32℄ for an illustration of differing forms of a tive learning.

1 Per eption-A tion Learning as Model for Self-Updating Cognition 7of spontaneously inferring both the most appropriate per eptual model by whi hto interpret the agent's external surroundings, as well as the most a urate modelof those surroundings in terms of the hosen per eptual framework. This environ-mental model, for a �rst-order-logi -based per eption-a tion learner, then onsistsin a parameterized, lause-based des ription of the subset of the total motor-spa ethat onstitutes the set of legitimately performable a tions, a ording to some setof very generalized physi al riteria (in our ase the performability and stabilityof the proposed a tion). Updating the per eptual framework will onsequently in-volve the reparameterization of the per eptual variables in this a tion-based �rst-order-logi model in order to allow representation of the environment in the mosta tion-relevant manner, permitting simultaneous empiri al re�nement of both theagent's world-model and its per eptual framework.Typi ally, be ause per eptual re-dundan y is progressively eliminated, a tions are arried out at in reasingly higherlevels of symboli abstra tion as the me hanism pro eeds.1.1.2 Implementing Cognitive Bootstrapping in the Logi alDomainThe entral motivating fa tor of this work is thus the notion that spe ifying a si-multaneous riterion for the inferen e of both obje ts and per epts is illogi al, oreven paradoxi al, within lassi ally dualisti ognitivemodels (eg [4℄), sin e obje t-model errors an be arbitrarily subsumed within per eptual states and vi e-versa3.The potential for unintentional ' ognitive tautology' through re ipro ally de�n-ing per ept and obje t states hen e always exists in fully autonomous ognitiveagents apable of updating their ognitive fun tionality in response to their externalworldmodel.We have hen e argued that this an be over ome if there exists a stronga priori inter onne tion between per ept and motor states. However, in addition tothis per ept-motor inter onne tivity, the fully autonomous ognitive agent must, byde�nition, be apable of generalized representation of its experien e, sin e the a -tions triggered in relation to parti ular per eptions would otherwise have to be spe -i�ed individually, in advan e. They would hen e not be onstitutive of the intention-ality of a tion required for true autonomy. Various approa hes to the generalizationof experien e within arti� ial ognitive agents are possible; for instan e, luster-ing [26℄ and subspa e representation [16℄. We fo us here ex lusively on �rst-orderlogi al representation, the simplest logi that permits quanti� ation over instan es.Variables are thus the entities within �rst-order logi that serve to unify possibilitiesof instantiation. Spe i� per eptions an hen e be represented as instantiations ofper eptual variables, and similarly, spe i� on�gurations of the obje tive domain an be onsidered as instantiations of the environmental variables.3 Indeed, for a non-intera ting ognitive agent, there are effe tively no meaningful autonomously-derived riteria that an be applied to per eptual inferen es in onsequen e of the fa t that nonon-arbitrary ost fun tion apable of distinguishing between rival per eptual hypotheses exists apriori ( f Quine [37℄ on the ontologi al underdetermination problem).

8 David Windridge, Josef KittlerObviously, this implies that the environment is sus eptible to hara terizationvia�rst-order logi al lauses. We ontend that this will always be true at some level ofthe per eption-a tion hierar hy, typi ally the very highest levels (where the level ofabstra tion of generalized per epts be ome su h that formalized symboli manipu-lation an be applied). Thus, although the human per eptual environment might behighly sto hasti and ontingent in the lower-levels (for instan e, salien y lustersin the visual �eld), higher level representations (eg 'ladder', ' ar') will, in general,be sus eptible to rule-based reasoning of the kind: 'the obje t ahead is a ladder and ould therefore be limbed', 'the obje t over there is a ar and ould thus be driven',et . The urrent investigation is hen e an attempt to demonstrate a me hanism fora hieving ognitive bootstrapping at the formal and relational level, with the testenvironment thus hara terized as entirely relational and without sto hasti ompli- ation. As su h, the method is intended to demonstrate the top-level of a tivity of anopen-ended P-A learning agent (the relationship between the formal and sto hasti learning is illustrated in [26℄, for a non-per eptually bootstrapping ognitive agent).By attempting the implementation of a tive per eption-a tion agen y within aproto ol-driven (ie rule-based) environment, we will hen e show that in arrying-out �rst-order logi indu tion of the proto ols that govern permissible a tions it ispossible to perform a remapping of the variable input/output stru ture of inferred lauses su h that a maximally ompa t set of variables governing a tion-legitima y an be obtained. Be ause of the per eption-a tion linkage impli it in the agent'sdesign, this variable remapping also serves to de�ne a maximally ompa t set ofper eptual variables through whi h the agent an reinterpret its external environ-ment, with important onsequen es for the learning algorithm as indi ated below.This remapping is arried-out in the manner indi ated earlier. Sin e, in per eption-a tion theory, a tions serve as the linkages or relations between individual per ep-tual states, one riterion for optimal per eptual representation is immediately sug-gested. This is the notion that per eptual states should be both exhaustively andunambiguously delineated by the hypothesized set of legitimate a tions, fAgeng,that have been previously determined by �rst-order logi al inferen e. It is hen epossible to de�ne a generalized set of per eptual states that are ideally orrelatedwith the inferred generalized a tion states through the appli ation of a onditionof bije tivity of a tion states with respe t to pairings of the proposed per eptualstates. (Re all that these pairings are ne essarily temporal in nature, being betweeninitial per eptual states, Pinitial, and �nal per eptual states, P�nal). Hen e, we havethat fAgeng, fP0genginitial�fP0geng�nal. This bije tivity riterion is appli able at allstage of the per eption-a tion hierar hy, serving as a guiding onstraint throughoutthe open-ended inferen e of novel per eptual frameworks and obje t-entities.The notion of bije tivity between per eption and a tion an be thought of as aformalization of the phenomenologi al theory that per eption dire tly relates to, andindeed, expresses a tion possibilities ( f in parti ular [18℄). Similarly, the notion ofaffordan e (eg [13, 28℄), urrently the subje t of mu h attention within ognitives ien e, proposes that what is per eived in the external world is not simply the inertset of spatial relations that hara terize geometri obje ts. Rather, what is per eivedis a set of entities hara terized in terms of the a tivities and manipulations that an

1 Per eption-A tion Learning as Model for Self-Updating Cognition 9observing subje t an perform with them. Both of these s hools of thought hen eattempt to de�ne a middle ground within the lassi al distin tion between subje tsand obje ts by introdu ing the notion of embodied, a tive agents whose per eptualapparatus is an intrinsi part of the environment that they seek to des ribe4.The maximally- ompa t des riptivity of legitimate a tions in the obje tive do-main onsistent with the bije tivity riterion thus onstitutes one half of the pro-posed strategy for the optimization of per eptual representation. A ting in paral-lel with this is the opposing per eptual onstraint of retaining maximal agen ywithin the environment, su h that the agent must be apable of per eptually dis-tinguishing the onsequen es of all of the possible legitimate a tions determined by�rst-order logi al inferen e. The result of these two onstraints is hen e a riterion apable of over oming the Quinean [37℄ problem of the arbitrariness of per ep-tual inferen e with respe t to observed data, whi h, unlike the standard problem ofobje t-model inferen e with respe t to observed data, is underdetermined for non-intera tive ognitive agents. In this s enario, optimal per eptual representation thusexpli itly maintains the link with optimal obje tive representation, yet retains suf�- ient distin tion from it to enable relatively independent ognitive updating without ompromising the 'obje tivity' of the per eptually-derived environment model.Implementing ognitive bootstrapping in the logi al domain will hen e �rstlyinvolve the use of �rst-order logi al indu tion to derive a generalized model of le-gitimate a tions, fgen, from exploratory samplings of the a priori a tion spa e. fgenthus has the form of a set of logi al lauses de�ned over the set of a priori per- eptual variables fPg = fP1;P2;P3 : : :g. Se ondly, a reparameterization, (fP0geng =fP01;P02; : : :g s:t: jP0genj � jPj), of the per eptual variables present in the inferred lauses will take pla e in order to ensure maximally- ompa t per eptual des rip-tivity of the permissible a tions via the bije tivity riterion. This latter aspe t of theapproa h (o upying the whole of Se tion 1.4) is thus the prin iple methodologi al ontribution of the urrent investigation.1.1.3 Generalized Obje t Classes in Cognitive Bootstrapping: TheAffordan e/S hemata Distin tionThe affordan e/s hemata distin tion lies at the heart of ognitive agen y, and an beunderstood in this investigation in the following way: Affordan es are those possi-bilities offered by obje ts in the world in relation to the agent's motor apabilities.As su h, they o upy an intermediate position between the lassi al Cartesian polesof subje t and obje t. However, they are unquestionably empiri al fa ts (fa ts re-lating to the intera tion of agent and environment), in the sense of being falsi�ablea ording to the standard Popperian [36℄ riterion of empiri ism.4 More general arguments for the importan e of embodiment to ognition an be found in [1,9,12,24,25℄.

10 David Windridge, Josef KittlerS hemata are high-level representations of a tion possibilities predi ated on as-sumptions about the affordan es offered by the world (the agent's world-model is hara terized as the total set of assumed affordan es). Hen e, be ause the ogni-tive bootstrapping agent is intended to produ e novel per epts in relation to its ex-ploratory �ndings about the world, s hemata for employing these inferred per eptsmust also be apable of falsifying them.An example of this falsi�ability might hen e be the postulation of a novel per- eptual ategory, 'ladder', on the basis of ertain salien ies, or aggregate salien iesin the agent's visual �eld. In a P-A framework, however, this per ept is further hara terized by its assumed affordan e of agent a tion possibilities (su h as theagent's being able to rea h-out and grasp the individual rungs of the ladder). But inorder to justify its hara terization as a single per eptual entity within a per eption-a tion framework, the ladder must also have a singular fun tion in terms of the thes hemata for its employment. In this ase, the singular fun tion is ' limbability',in the sense that an entire ladder must be utilized to a hieve this. The per eptual ategory 'ladder' is thus falsi�able if it turns out that the ' limbing' s hemata isunful�llable (perhaps the rung materials in agent's world are too weak to support limbing), in whi h ase the P-A-based segmentation of 'ladders' in the world is re-dundant. All postulated per eptual ategories in a P-A frameworkmust be similarlyfalsi�able via their asso iated s hema.In ognitive bootstrapping terms, s hemata are thus abstra tions of the exist-ing per eption-a tion hierar hy that (if they a hieve a suf� ient level of empiri al on�rmation) an themselves be ome affordan es by virtue of being linked to per- eptual inferen e. (The identi� ation of the per ept ategory 'ladder' is dire tly orrelated with the s hemata ' limb'). The per eption-a tion hierar hy is thus end-lessly extensible, with the inferred s hemata be oming in reasingly abstra t (su hthat, for instan e, they an be treated in formal logi al terms).Inferred high-level level per eptual representations in the ognitive bootstrappingagent must therefore always have this hara teristi of hierar hi al dependen y onlower-level per epts. The per ept-a tion link always ensures that these per eptualrepresentations are orrelated with high-level a tions (su h as limbing), that are inturn built on lower-level a tion apabilities (in this ase, the basi obje t manipu-lation apability): The following paper an be seen as an attempt to formalize thisidea at the level of logi al indu tion. The formation of generalized obje t lasses viaper eptual reparameterization an hen e be seen as a means for forming high-levels hemata in relation to the affordan e possibilities of the environment.1.1.4 A tive Learning and Cognitive BootstrappingPer eptual inferen e of the type detailed above, while signi� ant in over oming anumber of on eptual dif� ulties asso iated with autonomous, self-updating ogni-tive agen y, would be of only abstra t interest unless it an be shown to have useful onsequen es in the obje tive domain. However, it is lear from the above that, in

1 Per eption-A tion Learning as Model for Self-Updating Cognition 11a per eption-a tion ontext, the postulation of a novel set of generalized per eptualvariables also impli itly suggests a set of generalized a tion states. In performingthe per eptual remapping (fPg! fP0geng), the agent hen e narrows-down its hoi eof potential exploratory a tions to those onsistent with the notion of testing its in-ferred model of the environment.Thus, while the non-bootstrapping per eption-a tion agent must arry out explo-ration via random sampling of the a priori motor-spa e, the same agent employingper eptual updating of the type indi ated instead samples the inferred a tion spa erandomly. This has the natural orollary that a priori motor-spa e is sampled a -tively ( f eg [3, 32℄), exploration being ondu ted on the basis of the agent's per- eptual inferen es. A tive learning is hara terized by the intentional sele tion oflearning examples: it is typi ally used in environments in whi h there is an abun-dan e of unlabeled data and where labeling data is ostly. This is hen e true of theP-A learning environment in that the determination of the su ess of an exploratorya tion (ie, its labeling as 'su essful' or 'unsu essful') is intrinsi ally expensive intemporal terms. If we know that a broad lass of a tions are likely to be unsu ess-ful, it is hen e not ne essary to sample them at the same rate as the generally mu hsmaller lass of su essful a tions. Consequently, the fa t that ognitive bootstrap-ping renders unper eivable (at the highest level of representation) those a tions thatare not onsistent with the agent's estimation of what is possible in a tion terms,implies that exploration a quires an a tive aspe t.Thus, although it is possible for the agent to a hieve a similar sort of a tive learn-ing without per eptual inferen e by randomly sampling the a priorimotor-spa e and he king for onsisten y with the inferred model of legitimate a tions5, the fa t ofhaving reparameterized the per eptual spa e in a more ompa t formmeans that we an dire tly suggest exploratory a tions without the omputational inef� ien ies of onsisten y he king the samples. This is then an a tive learning strategy similarto that of [39℄, but arising naturally within the ontext of per eption-a tion learn-ing as a result of having determined an empiri ally onsistent and non-paradoxi almodel for per eptual updating. Thus, a roboti learning agent that initially sets out tosto hasti ally sample the a tion-domain 'move the gripper from position X to posi-tion Y', would, under an appropriate per eptual reparameterization, instead sto has-ti ally sample the a tion-spa e 'move the obje t A onto the obje t B'. It thereby'intentionally' sele ts hypothesis-spe i� learning examples in the manner requiredof the a tive learning pro ess. These samples will then be su h as to more read-ily on�rm or refute the per eptual hypothesis that the agent's world onsists in'movable obje ts'. They will also more readily refute the external obje t hypothesis(namely, that the spe i� per ept experien ed by the agent is in fa t a movable ob-je t within the world). Hen e, a ognitive bootstrapping system is simultaneouslyan a tive learning system with respe t to both the obje t and per ept hypotheses.We hen e envisage an iterative, three-stage pro ess of a tive-learning onsist-ing of: (1) Randomized exploratory a tivity arried-out in terms of the urrently-assumed per eption-a tion model; (2) Indu tion of a novel a tion legitima y model5 Though, importantly, without the same guarantee of uniform sampling of the legitimate a tions(see Con lusions).

12 David Windridge, Josef Kittlerin terms of the urrent per eptual framework; (3) Reparameterization of the per- eptual framework in order to most appropriately (ie most ompa tly) represent theinferred a tion-model.Note that this a tive exploration by random sampling of the remapped per ep-tual variables is only possible be ause of the hierar hi al link that exists betweenthe inferred per ept spa e and the a priori per ept spa e (the hierar hi ality arisingfrom the fa t that the former spa e is a subset of the latter). As su h, it representsa 'higher', more symboli level of representation of the sensory data (we shall latergive an intuitive example of this). Cognitive bootstrapping in the manner des ribed an therefore be onsidered a method for the spontaneous generation and groundingof symbols in the manner of Harnad [17℄. A tive exploration, in driving explorationfrom the highest, most symboli level of the hierar hy, hen e serves to ontinuouslyre-train all of the hierar hi al levels existing beneath it. Though beyond the s opeof this paper to demonstrate, the method thus has, as a hierar hi al reinfor ementlearner [7℄, the apa ity to update itself in a manner apable of robustly a ommo-dating environmental hanges.The most prominent aspe t of the hierar hi ality inherent in ognitive bootstrap-ping demonstrated in the following paper, however, is that brought about by theprogressively higher levels of per eptual representation. Thus, despite the fa t thatexploration is always undertaken sto hasti ally, randomly- hosen a tions within theabstra ted, top-level per ept domain in reasingly take the form of intentional a -tions when onsidered in terms of the a priori motor spa e.1.1.5 Stru ture of InvestigationIn setting out our approa h to ognitive bootstrapping within the relational domainand demonstrating its utility within a simulated logi al environment, we shall stru -ture the investigation as follows: Se tion 1.2 is on ernedwith the nature of the test-environment and the agent's relation to it in terms of its a priori per eption-a tion apabilities. It also sets out the means by whi h these are des ribed in �rst-orderlogi al terms. Se tion 1.3 deals with me hanism by whi h the generi ognitiveagent is to infer the spe i� logi al rules underlying its environment given onlythe out omes of exploratory a tions. Se tion 1.4 then des ribes how the ognitivebootstrapping agent an utilize this logi al inferen e to perform a remapping of itsper eptual spa e, su h that apparently random exploration of its environment on-stitutes an a tive learning approa h with respe t to environmental rule inferen e.Results of the appli ation of this te hnique with respe t to a ben hmark passive-learner then onstitutes Se tion 1.5 of the paper. Se tion 1.6 �nishes by dis ussingthe impli ation of these results both in pra ti al terms, and in terms of the impli a-tions for autonomous ognitive agents apable of meaningfully updating their own ognitive apparatus at the same time as their models of the external world.

1 Per eption-A tion Learning as Model for Self-Updating Cognition 131.2 A Simulated Environment For A tive Per eption-A tionLearningFollowing the pre eding des ription of P-A learning in abstra t terms, we shallhen eforth �nd it onvenient to introdu e the proposed ognitive-bootstrappingmethod within a on rete ontext, and only after whi h will the generalization toarbitrary environments be dis ussed. We are hen e here interested in ognitive boot-strapping only at the relational-levels of the per eption-a tion hierar hy, in whi han idealized, dis rete representation of the environment is already assumed to ex-ist. It is hen e lear that the domain in whi h we seek to implement the pre edingideas should be one for whi h there exists a lear, proto ol-driven riterion of a tionlegitima y. This legitima y should obviously be re�e tive of the onstraints of ana tual physi al environment, albeit at a suf� iently abstra ted level. Hen e, in orderto generate a preliminary instantiation of the notion of ognitive bootstrapping, wesele t a simulated 'shape-sorter' puzzle as the domain of agent a tivity.Within the simulated shape-sorter, variously shaped puzzle pie es may be ar-bitrarily transported around a three-dimensional volume via a 'gripping arm' (thathen e onstitutes the a tive omponent of the arti� ial agent). Other entities exist-ing within the a tive arena in lude the surfa e of the puzzle and, within this, a seriesof holes that orrespond uniquely to ea h of the shapes. The totality of these entitiesare assumed to rest on an impermeable surfa e. We further assume that the agenthas an idealized per eption of this environment (ena ted, perhaps, via hypotheti- al ameras positioned outside of the immediately a tive domain). Low-level visiontasks su h as obje t segmentation and three-dimensional re onstru tion are hen eassumed to have been �awlessly arried out at a hierar hi al-level prior to that inwhi h ognitive inferen e is to take pla e, su h that no per eptual errors exist at thislevel (this arti� ial restri tion is in no way a prerequisite of the method; rather it isa simplifying assumption).1.2.1 The A Priori A tion Spa eWithin this simpli�ed environment the potential range of a tions available to theagent ( orresponding to the a priori motor spa e) are thus the positional transla-tions of the gripping arm, whi h is assumed to perform a 'grasping' a tion at theinitial stage of the attempted translation and a 'releasing' a tion at the �nal stageof the proposed translation. At this stage we do not, for simpli ity of inferen e, onsider the possibility of expli it obje t rotation, although the on ept of obje torientation is of relevan e, as we shall see. A tions are hen e spe i�ed via a six-tuple instru tion 'move(x1;y1;z1;x2;y2;z2)' orresponding to the transition fromposition-ve tor (x1;y1;z1) to position-ve tor (x2;y2;z2) within the a tive volume(whi h equates in physi al terms to the three-dimensional spatial range of the grip-ping arm).

14 David Windridge, Josef KittlerHowever, the existen e of an a tion state within the a priori motor-spa e is nota guarantor of a tion legitima y, a notion that is required if there is to be mean-ingful intera tion between the agent and its environment6. De�ning an appropriate riterion for the legitima y of a tions within the environment an, in a sense, be onsidered a form of agent meta-goal spe i� ation, though one that is minimallyrestri tive with respe t to the agent's potential exploratory a tivities.A tion legitima y in the shape-sorter environment is hen e determined by thefeasibility, stability and utility of the proposed transition. The �rst of these on-straints, feasibility, is determinedby the physi al requirementof the non- oin iden eof obje ts; one obje t annot be legitimately moved into another. The stability of aproposed a tion is determined by whether its intended �nal state would undergo anyfurther observable transition not initiated by the agent; situations in whi h a movedobje t is released without anything beneath it are hen e illegitimate. The stability ondition thus ensures the temporal reversibility of a tions, su h that the environ-ment an be des ribed in terms of relatively simple �rst-order relational predi atesthroughout the experimental pro ess. The set of transitions hen e has the losedmathemati al stru ture of a monoid. To this end, we also require that positions andlengths are dis retized to identi al unit-lengths, su h that partial overlaps betweenobje ts are not permitted. The �nal onstraint, a tion utility, refers to the notionthat the proposed transition should do a tual physi al work (that is, result in a per- eived environment hange) if it is to be onsidered legitimate. The gripper annottherefore simply transit from one uno upied position to another, though this is bothstable and feasible under the pre eding de�nitions.Hen e, if an instru tion move(x1;y1;z1;x2;y2;z2) is to be onsidered legitimatevia the above riteria, it must involve the gripping of an obje t lo ated at position(x1;y1;z1), followed by the release of the same obje t at a lo ation immediatelyabove the supporting surfa e provided by an unen umbered solid entity lo ated at(x2;y2;z2�1). A supporting entity an be any of those we have de�ned: the puzzle-base, another shape, or a hole. However, the latter entity is only supportive if itdoes not mat h the morphology of the moved shape, or if it does mat h the shape'smorphology, but has a differing orientation.It is onsequently possible to quantify the restri tions that this notion of a tionlegitima y pla es on the agent's generi , a priori motor spa e when it is embod-ied within the parti ular on�nes of the shape-sorter environment in the followingway. The initial motor spa e has a numeri magnitude given by (jxj � jyj � jzj)2,representing the ombination of initial and �nal lo ation possibilities, with jxj, jyjand jzj the respe tive ardinalities of the dis retized ordinal ve tors. Within thisspa e there is a onsistent quantity, jshapesj, of legitimately movable obje ts that an be pla ed on any suitable unen umbered surfa e. This supporting surfa e hasto ompletely bise t the volume in the z dire tion, but must not in lude the positionof the obje t itself7. Quantized in unit areas, the surfa e therefore has an average6 Indeed, in ertain phenomenologi al models [45℄, this differential in a tual and potential a tion apabilities onstitutes the agent's environment.7 Note that this supporting surfa e is different for ea h obje t, sin e differing obje ts slot intodiffering holes, with the holes a ting as support-entities otherwise.

1 Per eption-A tion Learning as Model for Self-Updating Cognition 15numeri magnitude of (jxj� jyj� p�j1j2), where p is the probability that a givenobje t is orre tly oriented with respe t to its orrespondinghole (for random initial on�gurations this is determined by the level of dis retization of the orientation pa-rameter, spe i� ally its re ipro al). The fra tion of the agent's a priori a tion spa ethat an be onsidered legitimate within the parti ular environmental ontext of theshape-sorter puzzle is hen e: jshapesj� (jxj� jyj� p)(jxj� jyj� jzj)2 (1.1)1.2.2 The A Priori Per ept DomainAs well as the a priori a tion-spa e detailed above, the agent is also equipped withan a priori per eptual 'spa e' through whi h it (at least initially) interprets its en-vironment. The aim of ognitive bootstrapping is hen e to determine the subset ofthis per eptual spa e that most ef� iently delineates the urrent hypothesis as towhat onstitutes the legitimate a tion sub-spa e, but in su h a way as to onservethe empiri al falsi�ability of this hypothesis. As su h, the a priori per ept ate-gories employed for this purpose, like that of shape, do not, so far as the agent is on erned, yet have any a tion-determined meaning (whi h may vary throughoutthe bootstrapping pro edure). The nominal designations of the a priori per ept at-egories are thus, at this stage of the experimental pro ess, labeled purely for thepurposes of our omprehension. In full, these ategories are hen e as follows; po-sitional o upan y (ie, the dete tion of the presen e or absen e of an entity at aparti ular lo ation), shape awareness, hole awareness, an awareness of hole-shapemorphologi al equivalen e, angular orientation and spatial adja en y8. These baseper eptual ategories are hen e distin t from the per epts we shall later infer in thatthey may be regarded, in phenomenologi al terms ( f [20℄), as the in-analyti (non-separable) attributes of per eption, or qualia. Thus, 'red ir le' might onstitutea singular per eptual ategory following per eptual inferen e, with the individualqualia 'red' and ' ir ular' the relevant intrinsi ategories, or attributes, of per ep-tion. The per eption of spe i� entities hen e onstitutes, in an impre ise sense, the o-ordinatizing of the intrinsi per eptual ategories (we shall later utilize the bije -tion prin iple to �nd an expli it oordinatization of the manifold underlying the apriori per epts that is pre isely de�ned).Thus, for this demonstration, we have ele ted to regard shapes and holes as apriori, rather than as omposite per eptual entities. The agent's per eption of theseentities onsequently onsists (if there are no other attributes) in the returning of8 In a more realisti ally omplex ognitive environment it would be possible to eliminate the apriori awareness of the morphologi al orresponden e between holes and shapes by resolving itinto a suitable omposite of the two per eptual ategories of positional o upan y and spatialadja en y. In fa t, these are more like the true Kantian a priori per eptual ategories. (A prioriin the sense of their being empiri ally undis overable, being rather the onditions of empiri aldis overy).

16 David Windridge, Josef Kittlera parti ular index or label from the set of all entities of the same type. Shape andhole entities, are thus distinguished extrinsi ally, rather than via any intrinsi har-a teristi s they might have for a more omplex ognitive system, su h as humanper eption.Having thus given an indi ation how the a priori a tion and per ept spa es aredelineated, we an now turn to the means by whi h we are to implement this per- ept/motor environment in terms of �rst-order logi . In su h a framework, it is log-i al variables that will serve as the means for generalizing over per eptual entitiessu h that it be omes possible to express exploratory propositions dire ted at per ep-tions that have not yet been dire tly experien ed by the agent. The orrespondingme hanism for 'per eption' within the �rst-order logi al domain will hen e be en-vironmental predi ation arried out by the ognitive agent, in whi h the per eptsare the full range of possible o-instantiations of the per eptual variables.We thus lay the foundation for the use of indu tive logi programming (ILP)methods in order to determine generalized environmental rule hypotheses fromwhi h novel per eptual variables an be extra ted.1.2.3 Implementation of the Shape-Sorter Puzzle in First-OrderLogi In having opted for a �rst-order logi al des ription of the shape-sorter, it be omespossible to render the underlying onstraints of the environment as a set of physi- al proto ols. Attempted transitions within the a priori motor-spa e hen e be omelogi al propositions of the form move(x1;y1;z1;x2;y2;z2), that are true or falsea ording to their legitima y in terms of these axioms. (Hen e in a generi motor-spa e equipped with a set of a priori per eptual variables, [L℄, the a tion proposi-tions will be predi ates of the form a tionn([L℄1 ; [L℄2), with the numeri subs riptsdenoting initial and �nal per eptual states, respe tively). In hoosing to model thissystem in PROLOG, we �nd that the agent's attempted a tions be ome goal states,or theorems to be proved via �rst-order resolution.We are thus, in effe t, seeking to onstru t a semanti parser for agent a tions in terms of the shape-sorter axiom set.Hen e, des ribing, within PROLOG, the a priori ognitive ategories listedabove as predi ates with variable arguments, we render the per eptual ategories ofshapes, holes, hole-shape orresponden e, angular orientation and positional o - upan y as, respe tively: is shape(A;L1), is hole(A;L2), hole shape mat h(A;B),orientation(X ;O) and position(A;X ;Y;Z) (quanti� ation over variables is impli itin PROLOG).The variables ranging over these per eptual ategories are delineated as fol-lows: A (and B) represent positionally-determined entity labels (ie holes, shapes,the puzzle-base itself) if the subgoal position(A;X ;Y;Z) is ful�lled. X , Y and Zrepresent ordinal positions along the three spatial axes; O is an angle. The vari-ables L1 and L2 are label variables a ting over the sets fshape1;shape2; : : :g andfhole1;hole2; : : :g, and are thus subsets of the positionally-determined entity label

1 Per eption-A tion Learning as Model for Self-Updating Cognition 17 lass. (We shall later dis uss the ne essity of adopting this predi ate form in generalenvironments). Re all that notions like obje t and angle are not yet operationallyde�ned for the agent; they are, so far, only per eptually differentiated with respe tto ea h other. Hen e predi ates su h as 'is hole' and 'is shape' should only be re-garded as making elementary per eptual distin tions that will later a quiremeaningvia asso iation with spe i� a tion-possibilities. (Thus, the predi ate is hole might orrespond to an agglomeration of dark per eptual entities established as onsti-tuting a single lass via unsupervized lustering at a lower level of the per eption-a tion hierar hy). The predi ate names we have adopted �is hole�, �is shape�, et ,are purely to assist reader omprehension.The ognitive predi ates asso iated with the a priori notion of spatial adja en yare rendered as in x(X1;X2), in y(Y1;Y2) and in z(Z1;Z2) for the three respe -tive spatial dire tions. Hen e, in onsequen e of the dis retization of these axes, theterm in x(X1;X2) is satis�ed only when X2 = X1+ 1. (Sin e we do not yet, forsimpli ity, in lude the possibility of obje t rotation by the agent, there is no or-responding notion of angular adja en y; angles are represented by dis rete tokenswithout relational ontent).Higher-level on epts su h as the uno upied lo ation immediately above an ob-je t an hen e be represented by on atenations of the a priori ategories: in thisparti ular ase the on atenation; position(A;X ;Y;Z), in z(Z;Z1), not(position( ;X ;Y;Z1)).(In PROLOG, omma sequen es of this kind indi ate a simultaneous ful�llment re-quirement for ea h of the subgoal predi ates).Having trans ribed the a priori per ept/motor ategories into a format suitablefor logi al predi ation, it be omes possible to spe ify the onstraint rules for theshape-sorter environment in these terms. Any exhaustive des ription of su h pro-to ols must in lude logi al onstraints equivalent to the physi al notions of obje tnon- oin iden e, obje t persisten e under translation, obje t instability in the ab-sen e of supporting stru tures, and so on. While these notions are innate, or at leastintuitive to human agents, theymust be derived by our simulated agent via �rst-orderlogi al indu tion in terms of the a priori per ept ategories applied to the out omesof exploratory a tions.The simpli�ed rules governing the proto ols of the shape-sorter environment,whi h may be onsidered to a t as a very generalized supervisor to the simulatedopen-ended learning agent, are hen e rendered as the series of PROLOG lauses:move(X1;Y1;Z1;X2;Y 2;Z2) :�position(A;X1;Y1;Z1); is shape(A;L); in z(Z1;Z3);not(position( ;X1;Y1;Z3));not(position( ;X2;Y2;Z2));in z(Z4;Z2); position(B;X2;Y2;Z4);not(hole shape mat h(L;B)):move(X1;Y1;Z1;X2;Y 2;Z2) :�position(A;X1;Y1;Z1); in z(Z1;Z3); is shape(A;L1);not(position( ;X1;Y1;Z3));not(position( ;X2;Y2;Z2); in z(Z4;Z2);position(B;X2;Y2;Z4); is hole(B;L2);hole shape mat h(L1;L2);orientation(L1;O1);orientation(L2;O2);not(O1== O2):

18 David Windridge, Josef Kittlermove(X1;Y1;Z1;X2;Y 2;Z2) :�position(A;X1;Y1;Z1); is shape(A;L1); in z(Z1;Z3);not(position( ;X1;Y1;Z3)); position(B;X2;Y2;Z2);in z(Z2;Z3);not(position( ;X2;Y2;Z3)); is hole(B;L2);hole shape mat h(A;B);orientation(L1;O1);orientation(L2;O2);O1== O2:(For the sake of larity in the following arguments an additional iteration vari-able, T , that exists within themove and position predi ates in order to delineate sep-arate temporal stages [iemove(T;X1;Y1;Z1;X2;Y2;Z2) and position(T;A;X1;Y1;Z1)℄is omitted throughout).Multiple lauses in PROLOG represent alternative possibilities for the satisfa -tion of the logi al onstraints. Clause one in the above rule-sequen e hen e rep-resents the possibility of pla ing shapes onto support surfa es that are not holesmat hing the shape's morphology. Clause two represents the possibility of pla ingshapes onto holes that do mat h the shape's morphology, but whi h have differingorientations. Clause three represents the possibility of pla ing shapes into holes thatboth mat h the shapes' morphology and have identi al orientations.This latter rule represents the lassi al 'solution-state' of the shape-sorter puzzle.Thus, although we will not expli itly hardwire any goal-seeking hara teristi s intothe agent, we shall later see that a tive testing of generalized rule inferen es relatingto the pla ing of shapes on top of surfa es (su h as the �rst sparse random samplingof the environment is likely to give rise to) an result in agent behavior that is su-per� ially similar to that implied by expli it goal-seeking. We might also spe ulatethat this observation applies more generally; the uniqueness (in the sense of being anex eption to the general rule) of the solution-state in a typi al proto ol-based puz-zle environment would render its salien y for an a tive-learning agent mu h greaterthan would be the ase for a passively-learning logi al agent employing indu tivemethods.1.3 Approa h to First-Order Logi al Indu tionThe �rst task of the ognitive bootstrapping agent, prior to any per eptual updating,is to infer the generalized rules governing the a hievability of the exploratory a -tions under onsideration. That is, the agent must attempt to derive the above lausestru ture (a ting in the apa ity of supervisor within the simulated environment)from spe i� instan es of those rules' appli ation. This is the task of Indu tive Logi Programming [33℄.The indu tive logi al approa h that is most readily appli able to our environmentis Muggleton'sPROGOL [34℄. PROGOL pro eeds in a top-downmanner, onstru t-ing themost spe i� lause (see below) overing the �rst positive exploratory exam-ple from a series of predi ate mode de larations. Mode de larations de�ne the typeand number of a predi ate's arguments, as well as its variable input/output stru -

1 Per eption-A tion Learning as Model for Self-Updating Cognition 19ture; they also determine the number of permissible instantiations of the predi atevariables. Sin e lauses in PROLOG are de�ned by head and body predi ates, bothtypes of mode de laration are required in PROGOL. A typi al body mode de lara-tion is hen e of the form:: � modeb(n; p(+A;+B;�C;�D)), with n the number of permissible instanti-ations, p the predi ate fun tor, [A;B℄ the set of input variables and [C;D℄ the set ofoutput variables.The most spe i� lause is hen e the lause with the most exhaustive predi a-tion onsistent with the mode de larations. As su h, it de�nes a lower bound of atheta-subsumption latti ewhi h has as its upper bound the empty lause (one lausetheta-subsumes another if and only if there exist a set of variable instantiations [iea substitution℄ su h that all atoms in the former lause will onstitute a subset ofthe atoms in the latter lause). The latti e itself is navigated in a general-to-spe i� fashion via heuristi ally-guidedA? sear hing. The heuristi in question is the lause ompressivity with respe t to the remaining positive examples insofar it is onsis-tent with the negative examples. The most effe tive of these is then sele ted as ba k-ground knowledge, and all positive examples that are rendered redundant by it areremoved from the total set of examples, after whi h the pro ess begins again withthe �rst of the remaining positive examples. PROGOL thus arrives at its estimate ofthe most ompressive lause, or set of lauses, onsistent with the example data.The output of the attempted PROGOL indu tion of the above rule-set from theset of exploratory a tions is hen e the 'obje tive' model of legitimate environmentala tions. We now turn to the question of how the orresponding 'subje tive' modelof per eption appropriate to this obje tive model is to be derived and usefully em-ployed within the puzzle environment.1.4 A tive Learning Via Cognitive Bootstrapping UtilizingFirst-Order Logi al Indu tionWe have de�ned ognitive bootstrapping as the simultaneous inferen e of opti-mal obje t and per ept models in su h a way as to avoid problems of under-determination, with a onsistent empiri al riterion for both obje t and per eptmodel sele tion sustained throughout the learning pro ess. We have indi ated thatper eption-a tion learning provides a natural framework for this kind of updating,requiring that the per eptive apabilities of ognitive agents are determined by theira tive apabilities. Formalizing this notion as a ondition of bije tivity between per- eptual transitions and a tion states, we shall now set out to implement an iterativelearning system that alternates between the two omplementary phases of obje t-model inferen e and per ept-remapping.Per eptually-motivated exploration is thenthe intermediary linking the two phases of ognitive bootstrapping.The �rst of these phases is hen e the attempted �rst-order logi al inferen e, viaPROGOL, of the lauses that determine the behavior of the shape-sorter from theset of umulative exploratory a tions labeled as legitimate or illegitimate by the su-

20 David Windridge, Josef Kittlerpervising PROLOG lause-set. In the se ond phase, the remapping of per eptualvariables deriving from this inferen e will dire tly suggest, in onsequen e of thosevariables' generalized nature, a novel set of exploratory a tion possibilities that anbe employed to test the obje tive model. More spe i� ally, be ause the per eptualupdating arried-out in the se ond phase of ognitive bootstrapping is designed togive optimal representativity to the obje tive hypothesis derived in the �rst phaseof ognitive bootstrapping, the novel a tion possibilities suggested by the remappedper epts are pre isely those that are onsistent with the model (in onsequen e ofthe bije tivity ondition). The ompressivity riteria for the a eptan e of per epthypotheses ensures that these onsistent a tion possibilities are a essed in far moreef� ient manner than they were in the a priori per ept-motor spa e. The per eptualremapping thus, in effe t, re-parameterizes the model of legitimate a tion transi-tions in the most ompa t manner possible.Hen e, we see that in having des ribed a ognitive agent with generalizing log-i al indu tion apabilities under the ondition of per eption-a tion bije tivity, wehave impli itly des ribed an a tive ognitive learning system. In general terms, anagent that employs a tive learning is one that utilizes environmental hypotheses tomotivate exploration in order that a more rapid onvergen e on the orre t model an be a hieved ( f eg [3, 32℄). A passively learning agent, in ontrast, derives itenvironmental hypotheses from randomized exploration.We might therefore expe tthe ognitive bootstrapping agent to be inherently more ef� ient at environmentalmodel determination than a purely passive agent: we shall set out to test this hy-pothesis in se tion 1.5. It should be lear, however, that our ognitive model goesbeyond that of the typi al a tive exploratory agent in simultaneously seeking anoptimal remapping of the per ept domain, in effe t, a tively learning both the per- eptual and obje tive environment at the same time. We therefore now turn to themethod by whi h this per eptual remapping is a hieved.1.4.1 Remapping the Per eptual VariablesImposing the ondition of bije tivity between a tion states and per eptual begin-ning/end state pairings implies that there must exist a mapping (Pinitial�P�nal)! Asu h that the bije tivity ondition holds: ie, fPginitial�fPg�nal , fAg, where fPgis the omplete set of per eptions, and fAg the omplete set of a hievable a tions.Hen e, if, for the sake of example, we were to set about imposing this ondition onthe a priori per eptual spa e, we would pro eed in following way. The default a -tions in the absen e of any experimental data would have to be assumed to be thoseof the a priori motor spa e, move(X1;Y1;Z1;X2;Y2;Z2). However, the possibilityof spe ifying any individual (lower- ase) proposition,move(x1;y1;z1;x2;y2;z2), isalso an assertion of the existen e of an exhaustive set of singleton maps:f(x1;y1;z1)g! f(x2;y2;z2)g 8 x1;y1;z1;x2;y2;z2 (1.2)

1 Per eption-A tion Learning as Model for Self-Updating Cognition 21Hen e, omparing with the above per eption-a tion bije tivity ondition, andnoting that the (x1;y1;z1) range over the same spa e as the (x2;y2;z2) we see thatthe default per epts fPg are the set of positions, f(X ;Y;Z)g (ie o-instantiations ofthe X , Y and Z ordinates). In other words, given the a priori per eptual predi atestru tures available to the agent, su h as shape, orientation and so on, it is, underthe ondition of bije tivity, only the predi ate stru ture position that properly on-stitutes a per ept in the absen e of experimental onstraint data. (This re�e ts thefa t that a notion su h as shape does not yet have any a tion-determined per eptualsigni� an e).While this result is trivial (and to a ertain extent tautologi al) for the defaultmotor-spa e, it is indi ative of the approa h to per eptual updating adopted whenexploration gives rise to results that break the assumed equivalen e between the apriori motor-spa e and the set of possible a tions.Now, for the sake of demonstration, suppose that randomized sampling (ie ex-ploration) of the a priori a tion domain (f(X ;Y;Z)g�f(X ;Y;Z)g) has given rise toa suf� ient number of legitimate a tions (say, 5) for logi al inferen e to be ena ted.(This would imply around a two orders of magnitude greater number of illegitimatesamples - see equation 1.1). Further suppose that the appli ation of PROGOL to the umulative exploratory data has given rise to the inferen e of the partially a uratelegitima y rule:move(X1;Y1;Z1;X2;Y 2;Z2) :�position(A;X1;Y1;Z1); in z(Z3;Z2); position(B;X2;Y2;Z3);in z(Z4;Z1);not(position(A1;X1;Y1;Z4));not(position(A2;X2;Y2;Z2)):(whi h would orrespond to a onstraint-rule stating that only unimpeded obje tsmay be pla ed on top of other obje ts.)It is hen e apparent that, in attempting to �nd a rule that both generalizes andlegitimizes the training set, it has be ome ne essary for the inferen e system tointrodu e a set of new variables fA;B;Z3;Z4g beyond the existing six variablesfX1;Y1;Z1;X2;Y2;Z2g required to spe ify the a priori motor-spa e. It may be re- alled that the body mode de larations spe ify the input and output stru ture ofthe variables in the individual atomi propositions onstituting the inferred lause.Consequently, the atomi propositions an be represented as the nodes of a dire teda y li graph onstituting the full lause (a y li sin e we expli itly forbid re ur-sion in the inferred lause).Hen e, if we render the predi ate input/output stru ture of this rule visually (asin �gure 1.1), it is apparent that the six initial input variables in the a priori per eptspa e (ie, the three variables spe ifying the initial per epts (X1;Y1;Z1) and the

22 David Windridge, Josef Kittlerthree variables spe ifying the �nal per epts (X2;Y2;Z2)) are mapped onto only twooutput variables9.

PO

SIT

ION

NO

T(P

OS

ITIO

N)

INC

_Z

NO

T(P

OS

ITIO

N)

PO

SIT

ION

X1

Y1

Z1

−input −output

X2

Y2

Z2Z3

A

A1

B2

BFig. 1.1 Example s hemati of lause stru ture (see footnote 10).Consequently, if it were possible to de�ne predi ates in su h a way as to be ableto unambiguously reverse this variable mapping, it would be possible to de�ne a lause stru ture in whi h the input variables A and B uniquely de�ne the outputvariables [X1;Y1;Z1;X2;Y2;Z2℄, su h that the initial and �nal per ept states thatlink together legitimate a tions within the inferred model are preserved. (That is,so that the maps f(X1;Y1;Z1)g ! fAg and f(X2;Y2;Z2)g ! fBg are bije tive).It would, in so far as it is permissible to regard variable instantiations as ordinates,hen e be possible to 're-parameterize' the original six-dimensional per ept spa e asa two-dimensional per ept spa e hara terizing permissible a tions in the inferredrule. In intuitive terms, this new per ept spa e onsists of, respe tively the set of ob-je ts, fAg, and the set of surfa es, fBg (A being mapped onto any unimpeded obje texisting at lo ation (x1;y1;z1), and B being mapped onto any unimpeded obje tsexisting below the intended movement lo ation, (x2;y2;z2)). Thus we see that the ompressed per ept spa e represents a higher level of per eption than that of the apriori spa e, in the sense that it expresses relational, a tion-relevant abstra t on- epts that were not present in the default per ept spa e. Randomized exploration inthis higher level domain then onsists in the pla ing of random obje ts onto randomsurfa es, rather than simply the movement of the gripping arm between randomlo ations.In order to a hieve this remapping, it is ne essary to stri tly de�ne predi atesas bije tive fun tions between input and output variables, rather than merely on-straints upon them (both forms of predi ation are possible within PROLOG). Thismeans that agent predi ation must be on�gured su h that input variables relateto output variables via single instantiations10. Thus, when, for instan e, des ribing9 Note that the predi ates of the form not(: : :) do not generate output variables that an be mean-ingfully employed to address the per ept spa e, and are hen e indi ated by dotted lines in �gure 1to denote their removal from the remapping pro ess.10 The only permitted ex eption to this rule being predi ates with mode de larations that spe -ify only input variables (ie, those having the form modeb(1; p([+variabletype℄))), su h as

1 Per eption-A tion Learning as Model for Self-Updating Cognition 23the fun tional relation that exists between entity labels and three-dimensional po-sition ve tors, position(A;B;C;D), we have adopted a predi ate form that employsA as an output variable (a ting over obje t-labels) in su h a way that it is alwaysuniquely spe i�ed by the input variable triple, (B;C;D), ranging over ombinationsof the X ,Y and Z ordinates. This approa h is thus in expli it ontrast to the logi allyequivalent, but possibly more intuitive, des ription of positional o upation in termsof entity predi ates with forms of the type: hole(X ;Y;Z), shape(X ;Y;Z), et (whi hwould be either true or f alse as appropriate for the given positional input variables).By reje ting this predi ate form we are hen e here demanding that predi ates a t asfun tions between their internal variables, rather than a ting as hara teristi (orindi ator) fun tions from the internal variables to the Boolean set ftrue; f alseg.Signi� antly, all 'essen e'-like predi ation (of the type: has quality 1(X), has quality 2(X),et ) that a ts over a ommon variable (or set of variables),X , an be spe i�ed in su ha formby re-envisioning the various fun tor names (has quality 1;has quality 2; : : :)as spe i� lasses of sub-variable ranging over the instan es to whi h they apply.That is, we onvert the n hara teristi fun tions:has quality n(X)!ftrue; f alseg (1.3)to a set of disjoint subfun tions of the bije tive master fun tion:f : X !f L1st obje t with quality 1;L2nd obje t with quality 1; : : : ;L1st obje t with quality 2;L2nd obje t with quality 2; : : :g;su h that (has quality n(X ;L)! true) only when the two onditions that: (X hasthe quality n) and (L= f (X)) are ful�lled.In doing so, we obtain a novel set of predi ates, has quality n(X ;L), with therequired property of invertability, and whi h may be added to the mode-de larationsof the PROGOL ode in the form:: � modeb(1;has quality n(+X ;�L)). The revised predi ate thus be omes an ex-pli it subgoal for whi h a PROLOG interpreter must �nd, proof-theoreti ally, aninstan e of the output variable(s) satisfying the predi ate's logi al ondition in or-der to return a value of true. (Rather than simply returning the predi ate's truth valueas would be the ase for a hara teristi fun tion).It is hen e apparent, for essen e-like predi ation, that the magnitude of the setof output possibilities will always be less than that of the input set. More generally,however, any reasonable ognitive predi ate will have a predominately onstrainingeffe t on the set of input per eptual variables, sin e we expe t the initial, generi ,sensorimotor spa e to be limited by the spe i� situation in whi h the agent to whi hit belongs is pla ed. We therefore expe t that su h predi ates will typi ally maphole shape mat h(A;B). In these ases, the predi ate an be regarded as a variable-less termi-nating node of the dire ted a y li lause stru ture.

24 David Windridge, Josef Kittlerlarger sets of input variables to smaller sets of output variables. There are thus twoindependent me hanisms that a t to ompress the per ept spa e when we seek toinstantiate, in reverse, the inferred lause's input variables by the output variables.Hen e, in having de�ned a more ompa t but equally expressive per ept spa eby insisting on the reversibility of the lause stru ture, it be omes possible to ex-plore both the validity of this per eptual hypothesis as well as the environmental hy-pothesis represented by the inferred lause. It does this via per eptually-determinedexploration.Here, random instantiations of initial and �nal state pairings in the mod-i�ed per ept spa e orrespond to a tion propositions via the ondition of per ept-a tion bije tivity. Moreover, be ause the modi�ed per ept spa e ompa tly repre-sents only the a tion possibilities inherent in the inferred a tion-legitima y hypoth-esis, these a tion proposition serve as potential tests of the inferred rule. This is thenthe basis for the a tive omponent of the per eption-a tion learning of ognitivebootstrapping agents dis ussed in the next se tion.It should be noted that in the inferred lause that we have sele ted to illustratethe me hanism of per eptual remapping, there is a lear distin tion between ini-tial and �nal per epts in onsequen e of the existen e of the individual mappingsf(X1;Y1;Z1)g ! fAg and f(X2;Y2;Z2)g ! fBg. All reasonably a urate infer-en es will fall into this ategory (re�e ting the a priori independen e of [X1;Y1;Z1℄and [X2;Y2;Z2℄, and the temporal symmetry of permissible a tions within theshape-sorter environment).However, this is not ne essarily the ase, and it may thusappear that a strategy needs to be adopted to deal with rule inferen es that spuri-ously attribute a relationship between the variable sets [X1;Y1;Z1℄ and [X2;Y2;Z2℄(thereby attributing an a tion-independent relationship between initial and �nal per- epts). This is most easily a omplished by ensuring that all of the variables withinthe inferred lause's I/O stru ture (X1;Y1;Z1;X2;Y2;Z2;A;B;Z3 and Z4) respe tthis distin tion, reje ting outright any inferen es that do not. However, it is equallypossible simply to overlook the issue: this independen e of input and output per- eptual variables is only true of ertain per eption-a tion environments; other en-vironments, for instan e, where irreversible a tions are possible, may require �nalper eptual states to depend upon initial per ept states. Provided that all of the vari-ables governing the a priori a tion spa e are uniquely instantiated (whi h is guar-anteed by the lause reversibility ondition given above), it does not operationallymatter if the remapped per ept spa e temporarily relates initial and �nal per eptualstates. (Sin e the repeated random instantiations of exploratory a tions will even-tually reje t this supposition if it proves to be unwarranted). Thus, we expe t the�nal onvergent model of legitimate a tions within the shape-sorter environmentto respe t the independen e of initial and �nal per ept states, irrespe tive of whathappens during the learning pro ess.

1 Per eption-A tion Learning as Model for Self-Updating Cognition 251.4.2 Algorithmi Approa h To Per eptual RemappingWhen onstru ting a general lause to over a set of spe i� examples, PROGOLwill invoke new variables with novel labels as they are required in order to on-formwith the predi ate mode de larations. Consequently, the per eptual remappingpro edure re ounted in the previous subse tion an be des ribed via the in lusionrelationship that exists between the differing sets of variables fA1;A2; : : :Ang on-tained within the individual predi ates, predm(A1;A2; : : :Anm). Spe i� ally, if weex lude the possibility of set self-in lusion, and impose an additional dire ted edgebetween the input and output sets of variables within ea h of the predi ates, thenthe in lusion map onstitutes a dire ted a y li graph in whi h the sink verti es arethe a priori variables and the sour e verti es are the remapped per eptual variables.Determining whi h verti es are sour e verti es, and hen e establishing the set ofremapped per eptual variables for a given a tion-rule inferen e, is onsequently atrivial matter of as ending the in lusion hierar hy.However, it may be the ase that a number of the remapped per eptual variablesso derived range over identi al domains. When this o urs for predi ates ontainedwithin different lauses in the inferred rule-set, there exists a redundan y betweenthem by virtue of the lauses' mutual independen e. Hen e, for maximal om-pa tion of the per eptual variables, we an map the redundant variables onto ea hother. This amalgamation is a omplished by the set union operation ondu ted overvariable type. (Variable types are designated in the body mode de larations; for in-stan e, the entity and angle in the de laration 'modeb(n;orientation(+entity;�angle))). Hen e, suppose that we obtain a set of a lauses denoted fC1; : : : ;Cag, su h thatea h C ontains the set of remapped per eptual variables fA11;A12; : : : ;A1nag. We arethus presented with a series of potentially surje tive mappings between the sets ofremapped per eptual variables ontained within the lauses and the set of variabletypes, fT1;T2 : : :Tng;C1 : fA11;A12; : : : ;A1n1g ! fT1;T2 : : :Tng;C2 : fA21;A22; : : : ;A2n2g ! fT1;T2 : : :Tng;: : : ! : : :Ca : fAa1 ;Aa2 ; : : : ;Aan3g ! fT1;T2 : : :Tng; et (1.4)We onsequently propose to onstru t the minimal superset:M = n[y=1(M :M =maxx �����([t [Axt :Cx(Axt ) = Ty℄)�����) (1.5)with the apability of olle tively addressing ea h of the independent per ept spa es(ie su h that Cn �M ; 8n).

26 David Windridge, Josef KittlerShould the superset so formed be less ompa t than the set of a priori per eptvariables (that is, have a larger ardinality), then it is reje ted over the original per- eptual variables. In general, however, this is not the ase, and the new spa e is sub-stantially more ompa t. Note, that the methodwill not, in general, a hievemaximal ompa tion, given that for fairly omplex rule inferen es only some of the onstantswithin any variable type will be appli able. Individual samples in the ompa t spa eare hen e required to be tested a ording to the inferred rule before a tual, embod-ied exploration; however, the pro ess is very mu h a elerated by the redu tion insample-spa e dimensionality.As a alternative to equation 1.5, it is possible to treat the lauses sequentially bysampling in their individual per eptual spa es over alternating exploratory y les.However, we adopt the above method in order to generate as oherent a higher-levelper eptual domain as possible.Thus, in summary, we see that in ompa tly remapping the a priori per ept spa eon the basis of the inferred rule of a tion legitima y, we de�ne a spa e onsistingof high-level, a tion-relevant on epts apable of mediating between the agent'smotor possibilities and the physi al restri tions of the environment. We have, inother words, de�ned a set of affordan es.The key logi al onstraint on PROGOL for a hieving this, whi h is appli ablein any non-re ursive logi al environment in whi h per eptual variable instantiationsare linked via a tion-variable instantiations, is thus that of fun tional predi ate bi-je tivity. Thus, if a head mode de laration (representing an a priori a tion) links oneinstantiation of per eptual variables to another;:� modeh(1;a tion(+[per eptual variables℄1 ;+[per eptual variables℄2)),we an always ensure per eptual ompressibility in individual lauses by insist-ing that body mode de larations are of the form;:� modeb(n; p(+[inputvariablelist℄ ;� [out putvariablelist℄)), su h that n= 1.This an always be a hieved (see earlier) by su h means as writing entity-qualitydesignations in the form (has quality n(X ;L)! true), rather than as hara teristi fun tions for individual qualities. Per eptual variables that are ontained in multi-ple lauses an then be ompressed by appli ation of equation 1.5. Be ause onlya fra tion of the generi a priori a tion-spa e is employed, this enfor ed predi atereversibility always maps the a priori per eptual domain into a higher-level one viathe dire ted a y li graph stru ture impli it in logi al lauses. (Reversible predi ates[ie those with uniquely instantiated mappings between input and output variables℄ an always be treated as graph nodes).Hen e, generi appli ation of the method requires only that we spe ify the setof predi ates pertaining to the most basi per eptual ategories: in parti ular posi-tional label attributions ( olors, shapes, textures, et ) and spatial adja en y relations.(More general predi ates obtained by sto hasti lustering at lower levels of the hi-erar hy an also be added in the same manner). The relational ognitive bootstrap-ping method then ensures that omposite per eptual stru tures su h as 'surfa e' (ie,a omposite of positional o upan y, verti al adja en y, and positional va an y) arelearned as required: that is, only in so far as these are meaningful to the embodiedagent in a tion terms.

1 Per eption-A tion Learning as Model for Self-Updating Cognition 271.4.3 The Different Phases of A tive ExplorationIn de�ning a per eptual spa e appropriate to those a tions deemed permissible bythe inferred shape-sorter proto ols, the ognitive bootstrapping agent should �nddata apable of falsifying the a tion hypothesis far more qui kly than would oth-erwise be the ase. (By default PROGOL requires only one instan e to falsify ahypothesis, though this threshold an be in reased to a hieve varying degrees oferror-sensitivity). Hen e, in advo ating the above method for a hieving empiri allymeaningful per eptual inferen ewithin an indu tive �rst-order logi framework,wehave de�ned an agent that in identally performs a tive learning in both the per ep-tion and a tion domains. However, this is not to imply that this is the most ef� ientor omplete approa h to a tive learning possible within the ontext of ognitivebootstrapping. To begin to approa h typi al a tive learning performan e improve-ments (that is, with onvergen e times of the order O(log(1=error)) as opposed toO(1=error) for idealized a tive per eptron learning models [5℄) it is ne essary toextend the a tive learning framework.The most immediate su h requirement is that ne essitated by the eliminationof lo al minima effe ts, whi h an arise when a subset of the totality of lausesdes ribing permissible a tions has been a urately inferred. Clauses are indepen-dently satis�able, and hen e exploration arried-out by a ognitive agent in termsof a orre tly inferred rule-subset ould mean that the agent would never experi-en e any per ept apable of falsifying these rules. This would then give rise to anin orre t impression that the agent had onverged on the �nal model of permissiblea tions (the only riteria for model a ura y available to the agent being those ofthe legitima y of its exploratory a tions). To ountera t this, it is ne essary to in-du e the agent to arry-out exploration outside of the inferred hypothesis. This anbe a omplished most straightforwardly via uniform random sampling of the a pri-ori a tion spa e during an additional passive learning phase. However, the relativenumber of moves required for ea h phase is likely to vary from s enario to s enario,depending on the extent to whi h orre tly inferred lauses onstri t the spa e ofexploratory a tions. Therefore, when, in the next se tion, we attempt to quantify the onvergen e advantage attributable to ognitive bootstrapping, we shall employ avariety of passive-to-a tive exploratory move ratios in order to heuristi ally assessthe ideal strategy for per eptual bootstrapping in the shape-sorter domain. In havingto de ide upon a poli y that optimizes exploration with respe t to both the knownand the unknown per eptual spa es, the ognitive bootstrappingmethod thus resem-bles a lassi al reinfor ement learner [21℄, for whi h the analogous dilemma is theagent's attempts to maximize its environmental reward with respe t to the exploredand unexplored domains (although no form of ognitive inferen e is implied in thelatter ase).As well as the addition of a passive phase, it is also possible, in prin iple, tofurther re�ne the a tive phase so as to involve the dire t testing of a tion possibil-ities that have the apability to distinguish between ompeting obje t hypotheses.This is hen e lassi al a tive learning in the manner of [39℄. However, any su happroa h potentially ompli ates the interpretation of the learning agent in terms

28 David Windridge, Josef Kittlerof per eption-a tion theory, sin e ompeting hypotheses operate in differing per- eptual domains, and hen e exploratory a tions undertaken in any one of these do-mains would re�ne the model of a tion legitima y only with respe t to that per ep-tual model. Any su h system would thus not stri tly meet the riteria of ognitive-bootstrapping; the simultaneous derivation of environmental and per eptual modelsappropriate to the agent's a tive apabilities.Hen e, of these possibilities, we shall employ only the passive learningmodi� a-tion to the ognitive bootstrapping agent des ribed in the previous subse tion. Evenwithout these other modi� ations, the alternation between a tive and passive learn-ing produ es an exploratory method apable of as ending performan e gradientsduring the a tive phase, and (after undergoing random perturbations in the a pri-ori a tion spa e), a method apable of �nding alternative, potentially more global,gradients to as end during the passive phase. The approa h adopted might thus be onsidered a primitive form of simulated annealing [23℄.To provide a performan e ben hmark for this method, we shall also generatea purely passive learner in whi h PROGOL inferen e is applied to umulativeexploratory a tions derived from random sampling of the a priori a tion spa e(X1;Y1;Z1; X2;Y2;Z2). For both the bootstrapping and non-bootstrapping learn-ers, we shall umulatively apply logi al inferen e in bat hes of 10 exploratory a -tions for every learning iteration. For the ognitive bootstrapping learner this meansthat we alternate respe tively between the a umulation of sets of 10 and n� 10exploratorymoves via a tive and passive exploration: n is hen e an integer multiple ontrolling the ratio of a tive to passive exploration for the bootstrap learner.To initiate both the bootstrapping and non-bootstrapping learners, we employa bat h of 200 randomly-sampled exploratory moves in order to arrive at a po-sition from whi h a tive learning an ommen e. Hen e, we attempt �rst-orderlogi al inferen e only after suf� ient exploratory moves have been a umulatedto be reasonably ertain (that is, approximately 90% ertain) that rule inferen ehas taken pla e (randomly-sele ting 10% of the illegitimate rules for reasons ofef� en y). However, sin e it is not always possible to guarantee that this has o - urred, we revert, in the absen e of a generalized rule, to a passive exploration y le. A ura y �gures in this ase are nominally given by the legitima y rateof the random exploratory samples of the a priori motor spa e (ie, on average(jshapesj�(jxj�jyj� p))=(jxj�jyj�jzj)2), re�e ting the fa t that a tive learning inthe absen e of generalization would presumably ne essitate the re-ena ting of pos-itive examples. Under normal ir umstan es, however, a ura y is determined bythe error rate of the inferred rule (ie, the per entage of a tions in orre tly lassi�edas either legitimate or illegitimate) al ulated over the entire uniformly-sampled setof a tion possibilities within the a priori motor spa e.

1 Per eption-A tion Learning as Model for Self-Updating Cognition 291.5 Experimental ResultsThe result of the experiments for the passive and a tive learnerswith differing valuesof n (n = 1, 3 and 5, respe tively) are given in �gures 1.2, 1.3 and 1.4, represent-ing the average over 10 experimental runs (with standard deviations indi ated bythe error-bars). It an be observed that both the passive and a tive runs start froman initially high per entile a ura y as a onsequen e of the fa t that even a par-tially a urate rule-inferen e is suf� ient to orre tly eliminate the vast majority ofthe (jx1j� jy1j� jz1j)2 proposable transitions within the largely redundant a priorispa e.It is also evident in ea h of the three �gures that the a tive learning pro edurea hieves onvergen e onsiderably faster than the passive learning pro edure. If wede�ne onvergen e time as being the number of iterations required in order for per-forman e to fall within a given a ura y per entile, x, then the average onvergen etimes for the a tive and passive learners with respe t to n and x are those indi ated inTable 1 (n= ¥ de�nes a passive learner). In the most disparate ase, when n= 1 or5 and x= 99:0, this represents 9 iterations for the passive learner, and 23 iterationsfor the a tive learner; a 2:56-fold improvement in the onvergen e rate (see Table2).The respe tive absolute performan e values on whi h the learners onverge (de-�ned as the average performan evalue after the systems have ome within 1 per entof their maximum values) is 99:79 per ent for the a tive learner when n = 5 and99:38 per ent for the passive learner. Values for other settings of n and x are givenin Table 3. In all ases the absolute performan e at onvergen e is higher for thea tive learner. A ura y Per entilePassive/A tive Ratio x = 99% x = 98% x = 97%n= 1 9. 6. 3.n= 3 10. 4. 3.n= 5 9. 5. 4.n= ¥ (passive) 23. 7. 5.Table 1.1 Iterations Required To Rea h A Given Per entile A ura yA ura y Per entilePassive/A tive Ratio x = 99% x = 98% x = 97%n= 1 2.56 1.27 1.67n= 3 2.3 1.75 1.67n= 5 2.56 1.4 1.25Table 1.2 Bootstrap Learner Convergen e Rates As A Multiple Of The Passive Learner Conver-gen e Rate

30 David Windridge, Josef Kittler

0 20 40 60 80 100 120 14096

96.5

97

97.5

98

98.5

99

99.5

100

iteration number

accu

racy

(in

%)

active learner

passive learner

Fig. 1.2 A ura y versus iteration number for the ognitive bootstrapping and passive learners(bootstrap learner a tive/passive ratio=1).

0 20 40 60 80 100 120 14096

96.5

97

97.5

98

98.5

99

99.5

100

iteration number

accu

racy

(in

%)

active learner

passive learner

Fig. 1.3 A ura y versus iteration number for the ognitive bootstrapping and passive learners(bootstrap learner a tive/passive ratio=3).1.5.1 Alternative Experimental DomainIn order to assess the generalizability of these �ndings, we initiated a parallel set oftests in se ondary experimental domain with differing a tion and motor possibili-

1 Per eption-A tion Learning as Model for Self-Updating Cognition 31

0 20 40 60 80 100 120 14096

96.5

97

97.5

98

98.5

99

99.5

100

iteration number

accu

racy

(in

%)

active learner

passive learner

Fig. 1.4 A ura y versus iteration number for the ognitive bootstrapping and passive learners(bootstrap learner a tive/passive ratio=5). Per entage of Maximum AttainedPassive/A tive Ratio x = 99:75% x = 99:50% x= 99:00%n= 1 99.70 % 99.63 % 99.58 %n= 3 99.71 % 99.65 % 99.61 %n= 5 99.79 % 99.75 % 99.69 %n= ¥ (passive) 99.38 % 99.35 % 99.28 %Table 1.3 Convergen e A ura y (Mean A ura y After Having A hieved A Given Per entile OfThe Maximum A ura y Attained)ties, but (for the purposes of omparability) a similar per eptual predi ate stru ture.We thus employ identi al body mode de larations as before, but onstru t an al-ternative ground-truth lause, with a differing head mode de laration. The a prioria tion domain in this ase is hen e hara terized by a simulated robot arm withplanar movement potential along the X and Y axes only. However, it also has the apability to rotate any gripped entity to an arbitrary angle q around the z axis.Again, there is assumed to be a 'gripping' gesture at the outset of the a tion, and a'releasing' gesture at the end of the a tion. The a priori a tion ommand is hen emove(X1,Y1,q1,X2,Y2,q2,) for the initial and �nal motor states (X1;Y1;q1) and(X2;Y2;q2), respe tively.The domain of appli ation of this robot arm is an idealized 'jig-saw'-like envi-ronment in whi h any one of four puzzle pie es an be given a different positionor orientation on the puzzle-board subje t to one ondition. This is that the pie eis orre tly aligned with any other puzzle pie es that happen to be adja ent to it( o-aligned obje ts an pla ed adja ently in either the X or Y dire tions).

32 David Windridge, Josef KittlerThe ground-truth rule proto ols for this environment are hen e spe i�ed as thesimple four- lause sequen e:move(X1;Y1;q1 ;X2;Y2;q2) :�f ree position(X2;Y2); position(L;X1;Y1);orientation(L;q1 ;);in y(Y2;G); position(K;X2;G);orientation(K;P); theta2 == P:move(X1;Y1;q1 ;X2;Y2;q2) :�f ree position(X2;Y2); position(L;X1;Y1);orientation(L;q1 ;);in y(G;Y2); position(K;X2;G);orientation(K;P); theta2 == P:move(X1;Y1;q1 ;X2;Y2;q2) :�f ree position(X2;Y2); position(L;X1;Y1);orientation(L;q1 ;);in x(X2;G); position(K;G;Y2);orientation(K;P); theta2 == P:move(X1;Y1;q1 ;X2;Y2;q2) :�f ree position(X2;Y2); position(L;X1;Y1);orientation(L;q1 ;);in x(G;X2); position(K;G;Y2);orientation(K;P); theta2 == P:Predi ates are de�ned as before, albeit in two-dimensional terms (ie position(entity label;x ordinate; y ordinate). X , Y and q are quantized to 8,3 and 4 units, respe tively.The result of the experiments for the passive and a tive learners with differingvalues of n (n= 1, 3 and 5, respe tively) are given in �gures 1.5, 1.6 and 1.7, rep-resenting the average over 10 experimental runs (with standard deviations indi atedby the error-bars).It is again evident in ea h of the three �gures that the a tive learning pro edurea hieves onvergen e onsiderably faster than the passive learning pro edure. Con-vergen e times (de�ned as above) are given in Table 4 for three sample-points. Inthe most disparate ase, when n = 3 and x = 99:5, this represents 49 iterations forthe passive learner, and 7 iterations for the a tive learner; a 7-fold improvement inthe onvergen e rate (see Table 5). The respe tive absolute performan e values onwhi h the learners onverge (de�ned as the average performan e value after the sys-tems have ome within 0:5 per ent of their maximum values) is 99:86 per ent forthe a tive learner when n= 3 and 99:56 per ent for the passive learner. Values forother settings of n and x are given in Table 6. In all ases the absolute performan eat onvergen e is signi� antly higher for the a tive learner. However, it is the rateof onvergen e that we regard as being of prin iple signi� an e: while the a u-ra y per entage gives a good indi ation of the progress of rule indu tion, it doesnot do so in a linear fashion. In parti ular, the most signi� ant a tions (in terms ofthe spe i� ity of the proto ols appli able to them) are those relating to the pla -

1 Per eption-A tion Learning as Model for Self-Updating Cognition 33ing of a urately-aligned shapes into orrespondingly shaped holes: however, thesea tions o upy only a very small subset of the total exploratory domain.Results are hen e substantially better in this domain than in the previous ase.

0 20 40 60 80 100 120 14097.5

98

98.5

99

99.5

100

Iteration number

accu

racy

(in

%)

active learner

passive learner

Fig. 1.5 A ura y versus iteration number for the ognitive bootstrapping and passive learners(bootstrap learner a tive/passive ratio=1). Note that the �rst �ve points are omitted for graph-s aling purposes. A ura y Per entilePassive/A tive Ratio x= 99:5% x= 98:5% x= 97:5%n= 1 10. 4. 3.n= 3 7. 5. 3.n= 5 10. 5. 3.n=¥ (passive) 49. 7. 4.Table 1.4 Iterations Required To Rea h A Given Per entile A ura y1.6 Dis ussion and Con lusionsWe have hen e outlined a methodology for ognitive bootstrapping within a �rst-order logi al environment that enables the simultaneous inferen e of optimizedmodels of obje ts and per epts. The lassi al paradox asso iated with this type ofsimultaneous inferen e (namely, the potential meaninglessness of any empiri al ri-

34 David Windridge, Josef Kittler

0 20 40 60 80 100 120 14097.5

98

98.5

99

99.5

100

Iteration number

accu

racy

(in

%)

active learner

passive learner

Fig. 1.6 A ura y versus iteration number for the ognitive bootstrapping and passive learners(bootstrap learner a tive/passive ratio=3). Note that the �rst �ve points are omitted for graph-s aling purposes. A ura y Per entilePassive/A tive Ratio x= 99:5% x= 98:5% x= 97:5%n= 1 4.90 1.75 1.33n= 3 7.00 1.40 1.33n= 5 4.90 1.40 1.33Table 1.5 Bootstrap Learner Convergen e Rates As A Multiple Of The Passive Learner Conver-gen e Rate Per entage of Maximum AttainedPassive/A tive Ratio x = 99:75% x = 99:50% x= 99:00%n= 1 99.87 % 99.84 % 99.83 %n= 3 99.86 % 99.86 % 99.85 %n= 5 99.84 % 99.82 % 99.82 %n= ¥ (passive) 99.56 % 99.51 % 99.48 %Table 1.6 Convergen e A ura y (Mean A ura y After Having A hieved A Given Per entile OfThe Maximum A ura y Attained)terion for obje tive inferen e when the interpretation of empiri al data is affe tedby per eptual updating, whi h is itself determined via observation) is over omethrough the adoption of a per eption-a tion learning me hanism. In the expli it oupling of per eptions to a tions we thus aim to over ome the underdetermination(and hen e the arbitrariness) of per eptual updating present in Quinean-like [37℄models of per eption, for whi h sensory updating an only ever be arbitrary with

1 Per eption-A tion Learning as Model for Self-Updating Cognition 35

0 20 40 60 80 100 120 14097.5

98

98.5

99

99.5

100

Iteration number

accu

racy

(in

%)

active learner

passive learner

Fig. 1.7 A ura y versus iteration number for the ognitive bootstrapping and passive learners(bootstrap learner a tive/passive ratio=5). Note that the �rst �ve points are omitted for graph-s aling purposes.respe t to sensory eviden e. Another signi� ant advantage that stems from thisper eption-a tion approa h is the obviation of dif� ulties of framing [27℄, whi ho ur when per eptual data is a umulated in a manner that is not related to theagent's a tive potential.By formalizing the entral notion of per eption-a tion learning; 'a tion pre- edes per eption', as the requirement for a ondition of bije tivity between per ept-transitions and individual a tions existing within generalized models of a tion le-gitima y, we hen e provide a framework in whi h both per eptual and obje tivelearning are made possible. More simply, ognitive bootstrapping seeks to reatea spa e of a tion possibilities that are always per eptually realizable, and whereredundant a tion possibilities are eliminated from per eption.As well as eliminating redundan y, the ondition of bije tivity implies that per- eptual updates must retain the expressiveness of the a priori spa e, permitting ev-ery legitimate a tion possibility to be per eived, so that the determination of a tionlegitima y onstitutes the environment model hypothesis. The bije tivity onditionhen e allows us to eliminate dif� ulties typi ally asso iated with the hermeneuti ir le of interpretation [11, 38℄, in whi h per eptual updates suggested by exper-imental observation an progressively lose information about the environment intrying to a hieve the goal of per eived a ura y11. It is hen e ru ial that we retainthe a priori per eption-a tion spa e in order to ground the per eptual inferen es ina hierar hi al fashion.11 It is always possible to map per epts to smaller subsets su h that the ability to dis riminate'dif� ult areas' is lost, and falsifying a tion possibilities are no longer per eived.

36 David Windridge, Josef KittlerBeyond this, the expli it linkage of per eption to anti ipated a tion possibilitiesin per ept-a tion learning means that per eptual inferen es are also a tion infer-en es and vi e versa. Hen e, the appli ation of the ondition of per eption-a tionbije tivity to a generalized hypothesis-learner apable of ompa t representation ofits input-spa e (su h as PROGOL equippedwith the reverse variablemapping stage)implies that novel (that is, previously unexperien ed) a tion and per ept possibilities an be addressed by the learning system.Moreover, sin e su h novel a tion possibilities are ne essarily more onstri tedthan those of the a priori spa e, they have the apa ity, under random sampling, to�nd data apable of falsifying the obje t model hypothesis very mu h more rapidlythan would be the ase for random sampling of the a priori spa e. The ognitivebootstrappingme hanism we have des ribed is hen e also an a tive learningme h-anism.This investigation hen e set out, �rstly, to demonstrate that ognitive boot-strapping (simultaneous obje t and per ept inferen e) an be a omplished self- onsistently within a �rst order logi al environment via the expedient of remappingthe predi ate variable stru ture. Se ondly, it sought to demonstrate that su h a sys-tem, as an a tive learner, is apable of a hieving onvergen e on an obje tive modelfaster than a purely passive learner (ie, one that does not attempt per eptual updat-ing) when randomly sampling the a priori spa e.To this end, a PROLOG-based simulated shape-sorter environment was on-stru ted, in whi h �rst-order logi al inferen e was arried out via PROGOL, withpredi ate mode de larations so onstru ted as to allow for the reverse mapping ofthe per eptual variables. The typi ally more ompa t per eptual variable set arisingfrom this reverse mapping an then be randomly instantiated in order to provide aset of proposed experimental a tions in the a priori spa e for the ognitive boot-strapping agent to perform.In arrying out this experimental instantiation of ognitive bootstrapping in se -tion 1.5, we did indeed �nd that the simulated agent gave rise to signi� antly fastertraining (up to 2:56 times faster) than an equivalent, but non-bootstrapping agent byvirtue of this impli it a tive learning potential. In a se ondary test-environment thisperforman e improvement was at the seven-fold level.We should note that from a pra ti al perspe tive, would be possible, in prin iple,to bypass ognitive bootstrapping and a hieve a tive learning in the shape-sorter do-main by sampling exploratory a tions uniformly from the a priori six-dimensionalspa e (X1;Y1;Z1;X2;Y2;Z2), �ltering them for onsisten y with the inferred rule.However, it is mu h more omputationally ef� ient to �nd an appropriate param-eterization of the legitimate a tion-spa e denoted by the inferred rule. Moreover,it would not, in general, be possible to guarantee that uniform sampling within ana priori spa e would be orrelated with uniform sampling within an inferred per- ept spa e. This would apply if, for instan e, we were to hronologi ally relax theinstantiation uniqueness ondition on predi ate mode de larations, and permit re-verse mapping to be inje tive over time. Hen e, an a tion spe i�ed at the top levelof the hierar hy might now be instantiated after some number of a tion sub-stagesin the hierar hi al per eption-a tion level immediately beneath it (subgoals must

1 Per eption-A tion Learning as Model for Self-Updating Cognition 37hen e always have per eivable onsequen es). This latter possibility is more rep-resentative of biologi al ognition; inferred obje t-motor ategories su h as affor-dan es [13, 28℄ do not immediately onstrain every motor-possibility at the mus- ular level, but rather apply progressive onstraints hierar hi ally as different sub-per eptions and a tion possibilities be ome apparent. Hen e progressive hierar hi- al levels serve to ground the high-level a tion in the a priori sensorimotor systemexisting at themus ular-neuronal level by spe ifying sensorimotor sub-goals at ea hstage of the hierar hy.Beyond the exploration of this possibility, another obje tive of future resear hwill be to eliminate the requirement of predi ate mode de larations as they are ur-rently spe i�ed. Hen e, rather than having to assuming a given predi ate form be-fore inferring the lause variable stru ture (this being a limitation of PROGOL), wewould instead infer both the predi ate form and its variable stru ture. In this way,the method ould be made suf� iently generi so as to enable the agent to be pla edin a ompletely arbitrary environment with no prior assumptions beyond that of thea priori sensor-motor omplex.This possibility is very naturally en ompassed by the ognitive bootstrappingframework via its ability to utilize exploratory �ndings in high-level per eption-a tion domains to feedba k spe i� ation information to the lower levels in su ha way as to eliminate feature redundan y and feature under-spe i� ation. Hen e,suppose that we have resolved to treat per eptions as the means of differentiatinga tions and, furthermore, regard the ompressive apability of per eptual ompo-nents as being an essential omponent of ognition (after [49℄). It then be omesnatural to amalgamate those per epts that have logi ally-identi al a tion relationsat the a priori level. (Thus, for instan e, a ' hess-board'-pattered shape that wasinitially labeled via multiple per epts by the a priori image segmentation system,might instead be more logi ally addressed a single per ept after exploratorymanip-ulation has determined their onne tedness). Equally, and onversely, it be omesnatural for the ognitive bootstrapping me hanism to demand an expanded a prioriper ept domain when unable to disambiguate a tion out omes (for instan e, by re-�ning a priori image segmentation parameters in order to generate a broader rangeof per ept labels). Thus the a priori per ept domain presented to logi al indu tionneed have no signi� ant stru ture beyond that of a label, the a tion relations be-tween per epts serving as the sole means of determining obje tive models of theexternal environment. It is then possible to autonomously derive a predi ate of thetype 'is hole(A)' with exa tly equivalent hara teristi s to that hitherto spe i�edvia ba kground knowledge. This an be a hieved by exploiting the fa t that per- epts or ombinations of per epts apable of distinguishing holes from shapes (forinstan e olor-labels) an be oupled to the a tion-derived meaning of the on ept'hole' (dis overed by random exploration in the a priorimotor-domain) by general-izing over spe i� instan es in the usual manner. Groups of per epts that show thisproperty onsistently an hen e then be olle ted and ombined with the a priorimotor variables in order to spe ify novel predi ates for reuse via the �rst-order log-i al indu tion system. Su h an approa h may be thought of as the a tive sele tionof the feature-spa e underlying per eption, as distin t from the a tive sele tion of

38 David Windridge, Josef Kittlerthe operative subspa e underlying per eption as is the ase for the method we haveoutlined hitherto.More broadly, the ognitive bootstrapping method we have adopted, that uti-lizes per eption-a tion learning with bije tivity onstraints in order to provide aself- onsistent framework for per eptual updating, is potentially appli able to anylearning methodology apable of generalization and ompression. Thus, as well aslogi al quanti� ation over variable instan es, it is also possible to generalize overfeature-spa e ve tors via the usual me hanisms of statisti al pattern re ognition.Hen e, ognitive bootstrapping an equally be applied to per eptual variables gov-erning the parameters of a statisti al distribution within the sensory domain. For in-stan e, environmental lasses inferred via unsupervized learning may be utilized topropose agent a tions that result in a tive modi� ation of the parameters governingthe lustering behavior in a way that allows novel a tion-relevant obje t lasses to bederived. Here again, the problem of framing is resolved by adaptively �ltering-outper eived entities that have no bearing on the a tion apabilities of the agent, theseentities being a tively lassed as noise via the statisti al generalization me hanism.Su h a statisti al approa h to a tive per eptual reparameterization would almost ertainly need to be adopted for any real-world implementation of the ognitivebootstrapping agent: this is hen e our next resear h obje tive.Another issue of interest is in relation to supervised learning; a ognitive boot-strapping agent an learn not merely to imitate a tivity undertaken by the supervisor,but also to per eive the external environment in the same manner as the supervisor.In addition to the bene�ts asso iated with a tive learning, this alignment between theagent's and supervisor's per eptual spa es ould, for instan e, be used to bootstraplinguisti ommuni ation between the agents (in the manner of [41,42℄). Linguisti ommuni ation, in distin tion to other types of agent behavior, an trigger a tionsin response to sensory data referring to hypotheti al and/or generalized situations.As su h, the ommuni ating agent needs to ensure that its internal environmentalrepresentation orrelates with that of the ommuni ating agent. In the majority ofagent-based linguisti models this orrelation is given a priori. However, a ogni-tive bootstrapping agent is theoreti ally apable of learning a linguisti frameworkfrom the a tions of a trainer, so that linguisti models (su h as those governingwordreferen e) an be on�rmed via a mixture of exploratory and ommuni ative test-ing. Without this embodiment in the a tive domain, ultimately no de�nite assuran eof a ommonality of meaning between ommuni ating agents is possible (refer, forinstan e, to Wittgenstein's Philosophi al Investigations [48℄, or to Millikan's no-tion of biologi al ognition [30, 31℄). We hen e, again, note the importan e of theper eption-a tion framework in over oming the underdetermination of per eptual-updating in ognitively-autonomous agents: this onstitutes the prin iple motivationfor our work.

1 Per eption-A tion Learning as Model for Self-Updating Cognition 39A knowledgmentsThe work presented here was supported by the the EuropeanUnion, grant DIPLECS(FP 7 ICT proje t no. 215078)12Referen es1. Mi hael L. Anderson. Embodied ognition: A �eld guide. Artif. Intell., 149(1):91�130, 2003.2. R. Bowden, L. Ellis, J. Kittler, M. Shev henko, and D. Windridge. Unsupervised symbolgrounding and ognitive bootstrapping in ognitive vision. In Pro . 13th Int. Conferen e onImage Analysis and Pro essing, pages 27�36, September 2005.3. David A. Cohn, Zoubin Ghahramani, and Mi hael I. Jordan. A tive learning with statisti almodels. J. Artif. Intell. Res. (JAIR), 4:129�145, 1996.4. Cottingham, Stoothoff, Murdo h, and Kenney. The Philosophi al Writings of Des artes. Cam-bridge University Press, Cambridge, UK, 1991.5. Sanjoy Dasgupta, Adam Tauman Kalai, and Claire Monteleoni. Analysis of per eptron-baseda tive learning. In Peter Auer and Ron Meir, editors, COLT, volume 3559 of Le ture Notes inComputer S ien e, pages 249�263. Springer, 2005.6. J. Dewey. The re�ex ar on ept in psy hology. The Psy hologi al Review, (3):356�370,1896.7. Thomas G. Dietteri h. Hierar hi al reinfor ement learning with the maxq value fun tionde omposition. J. Artif. Intell. Res. (JAIR), 13:227�303, 2000.8. J.A. Fodor. The Modularity of Mind. MIT Press, Cambridge, MA, 1983.9. S. Gallagher. How the Body Shapes the Mind. Oxford University Press, Oxford, 2005.10. Ph. Gaussier. Towards a ognitive system algebra: A per eption/a tion perspe tive. In Euro-pean Workshop on Learning Robots, pages 88�100, Prague, Sept 2001.11. A. Geir. Explanation and Understanding: The Hermeneuti Ar . PhD thesis, University ofOslo Department of Philosophy, 2001.12. R. W. Jr Gibbs. Embodiment and Cognitive S ien e. Cambridge University Press, 2005.ISBN-10: 0521811740 � ISBN-13: 9780521811743.13. J. J. Gibson. The e ologi al approa h to visual per eption. Houghton-Mif�in, Boston, 1979.14. G. Granlund. Organization of ar hite tures for ognitive vision systems. In Pro eedings ofWorkshop on Cognitive Vision, S hloss Dagstuhl, Germany, 2003.15. G. H. Granlund. An asso iative per eption-a tion stru ture using a lo alized spa e variantinformation representation. In Pro eedings of Algebrai Frames for the Per eption-A tionCy le (AFPAC), Kiel, Germany, September 2000.16. G. H. Granlund and A. Moe. Unrestri ted re ognition of 3d obje ts for roboti s using multi-level triplet invariants. AI Magazine, 25(2):51�67, 2004.17. S. Harnad. The symbol grounding problem. Physi a D, (42):335�346, 1990.18. M. Heidegger. Being and Time. Bla kwell, 1996.19. S ott B. Huffman and John E. Laird. Flexibly instru table agents. Journal of Arti� ial Intel-ligen e Resear h, 3:271�324, 1995.20. E. Husserl. Ideas Pertaining to a Pure Phenomenology and to a Phenomenologi al Philosophy� First Book: General Introdu tion to a Pure Phenomenology. The Hague, 1982.21. �Leslie Pa k Kaelbling, Mi hael L. Littman, and Andrew P. Moore�. Reinfor ement learning:A survey. J. Artif. Intell. Res. (JAIR), 4:237�285, 1996.22. Immanuel Kant. Critique of Pure Reason. Cambridge University Press, 1999. ISBN:0521657296.12 However, this paper does not ne essarily represent the opinion of the European Community, andthe European Community is not responsible for any use whi h may be made of its ontents.

40 David Windridge, Josef Kittler23. S. Kirkpatri k, C. D. Gelatt, and M. P. Ve hi. Optimization by simulated annealing. S ien e,Number 4598, 13 May 1983, 220, 4598:671�680, 1983.24. G. Lakoff and M Johnson. Philosophy in the Flesh : The Embodied Mind and Its Challengeto Western Thought. Harper Collins Publishers, 1999.25. G. Lakoff and R. Nu�nez. Where Mathemati s Comes From: How the Embodied Mind BringsMathemati s into Being. Basi Books, 2000.26. D. Magee, C. J. Needham, P. Santos, A. G. Cohn, and D. C. Hogg. Autonomous learning for a ognitive agent using ontinuous models and indu tive logi programming from audio-visualinput. In Pro . of the AAAI Workshop on An horing Symbols to Sensor Data, 2004.27. J. M Carthy and P.J. Hayes. Some philosophi al problems from the standpoint of arti� ialintelligen e. Ma hine Intelligen e, (4):463�502, 1969.28. J. M Grenere and W. Ho. Affordan es: Clarifying and evolving a on ept. In Pro eedings ofGraphi s Interfa e 2000, pages 179�186, Montreal, Canada, 2000.29. M. Merleau-Ponty. Phenomenology of Per eption. New York: Humanities Press, 1962.30. R. G. Millikan. Language, Thought, and Other Biologi al Categories: New Foundations forRealism. The MIT Press; Reprint edition, De ember 1987. ISBN: 0262631156.31. R. G. Millikan. White Queen Psy hology and Other Essays for Ali e. The MIT Press; Reprintedition, Mar h 1995. ISBN: 0262631628.32. Andrew Moore. Variable resolution dynami programming: Ef� iently learning a tion mapsin multivariate real-valued state-spa es. In L. Birnbaum and G. Collins, editors, Ma hineLearning: Pro eedings of the Eighth International Conferen e, 340 Pine Street, 6th Fl., SanFran is o, CA 94104, June 1991. Morgan Kaufmann.33. S. Muggleton. Indu tive Logi Programming. A ademi Press, 1992.34. Stephen. Muggleton. Inverse entailment and progol. New Gen. Comput., 13:245�286, 1995.35. A. Newell and H. Simon. The theory of human problem solving; reprinted in ollins & smith(eds.). In Readings in Cognitive S ien e, se tion 1.3., 1976.36. K. Popper. The Logi of S ienti� Dis overy. (translation of Logik der Fors hung). Hut hin-son, London, 1959.37. W. V. O. Quine. Ontologi al Relativity. Columbia, 1977.38. P. Ri oeur. Hermeneuti s and the Human S ien es: Essays on Language, A tion and Inter-pretation. Cambridge: Cambridge University Press, 1981.39. H. S. Seung, Manfred Opper, and Haim Sompolinsky. Query by ommittee. In ComputationalLearning Theory, pages 287�294, 1992.40. G . Sommer, E. Bayro-Corro hano, and Th. Blow. Geometri algebra as a framework for theper eption-a tion y le. In Workshop on Theoreti al Foundation of Computer Vision, Ed. F.Solina, Wien, 1997. Springer Verlag.41. L. Steels. The origins of syntax in visually grounded roboti agents. In M. Polla k, editor,Pro eedings of the 10th IJCAI, Nagoya, pages 1632�1641. AAAI Press, Menlo-Park Ca.,1997.42. L. Steels and F. Kaplan. Bootstrapping grounded word semanti s. In Ted Bris oe, editor, Lin-guisti Evolution through Language A quisition: Formal and Computational Models, hap-ter 3. Cambridge University Press, 2002.43. R. Sun. Desiderata for ognitive ar hite tures. Philosophi al Psy hology, 17(3), 2004.44. R. Sun, T. Peterson, and E. Merrill. A hybrid ar hite ture for situated learning of rea tivesequential de ision making. Applied Intelligen e, 11:109�127, 1999.45. Bas. van Fraassen. The S ienti� Image. Oxford: Oxford University Press, 1980. ISBN0-19-824427-4.46. D Windridge and J Kittler. Cognitive bootstrapping: A survey. Te hni al report, CVSSP,University of Surrey, UK, 2005.47. D Windridge and J Kittler. Epistemi onstraints on autonomous symboli representation innatural and arti� ial agents. Appli ations of Computational Intelligen e in Biology: CurrentTrends and Open Problems, Studies in Computational Intelligen e (SCI), 122, 2008.48. L. Wittgenstein. Philosophi al investigations : the German text with a revised English trans-lation by Ludwig Wittgenstein. Oxford : Bla kwell, 2001. ISBN 0631231277.49. J G Wolff. Cognitive development as optimisation. In L Bol , editor, Computational Modelsof Learning, pages 161�205, Heidelberg, 1987. Springer-Verlag.