Collective Unintelligence: The Ground of Society

Helsinki ENSO ENSO 2013 - Castelfranchi

Collective Collective Unintelligence:Unintelligence:

The Ground of Society_____________________

Cristiano Castelfranchi

Institute for Cognitive Sciences and Technologies - Roma


Premise and IssuesA "cognitive" foundation"cognitive" foundation of social interaction and collective and macro-phenomena, that is, the identification of the needed mental mediators, the (individual or collective) "representations", behind the macro dynamics or relations, induces to a disappointing conclusion;

rather close to the idea of the unavoidable "alienation" of human powers, the unavoidable Leviathan face of Power, and the impossibility of direct and fully transparent demo-crazy ("government") of society.

Such conclusion is that: there are several structural and functional reasons at the

collective level why “we” have to be (individually and collectively) blind.

I will give a few but very crucial examples of that, providing the analysis of the mental counterpart of those socio-political objects.


Mind is not enough Mind is not enough Emergence, Self-

OrganizationFunctions and Cognitions:

examples

1


Mind is not enoughMind is not enough!!

the “individualistic + cognitive” approach is not sufficient for the social theory and processes (even when modeling joint and collective attitudes and actions).

The social actors do social actors do notnot understand, understand, negotiate, and plan negotiate, and plan for all their collective behavior and cooperative activity.

Computational & Cognitive Costs (“bounded rationality”)

and Negotiation and Transaction CostsCoordination Costs

would be unbearable!


Computational & Cognitive Costs (“bounded rationality”)

and Negotiation and Transaction Costswould be

unbearable!H. Simon has explained us why not only human intelligence is limited, bounded, and biased, but why it has to be so in principle.

Now we need a new Simon for explaining us at the collective level not just that

> collective intelligence is in fact biased, irrational, limited, as a result of individual intellectual limits and of socio-psychological phenomena (like mass psychology) but

> how and why it has to be so, also for functional reasons. In primis costs, uncertainty reduction, predictability, …


Some EXAMPLESSome EXAMPLES

“Spontaneous” Social Order

Social Functions

“Institutions” building

Normative Alienation

Power Delegation

Tutorial Relations


1. “Spontaneous” social order There are no alternatives to "spontaneous social order" (Hayek), self-organizationself-organization, that is, "the invisible hand" (A.Smith);

however, the achieved equilibrium may be bad, not optimal at all for the actors.

Not only bad, and not just "equilibrium" (like in the PD), but self-reproducing, and self-maintaining "orders", that is "ends" (A.Smith), "functions", impinging on and selecting our behaviors: "kako-functions".

It doesn't follow from the fact that each of us or each group wants the best, tries to maximize its utility, that the functional order be the best possible one.

We are not aware of the "ends" we follow (A. Smith).(below)


2. Social FunctionsWe play our social functions blindlyWe play our social functions blindly; not because they are unconscious, or because just based on reinforcement learning or on mere "habituses" (Bourdieu);

also our intentional and deliberated actions evaluated on their (visible and conceivable) consequences, "pursue" collective (bad) "ends".

If we would - for example - realize our irrationality and how marketing induces "needs", deceives and manipulates us, we couldn't play well our most crucial "role" in/for society: "consumers"!; supporting market and banks.


2. Social FunctionsFunctions install and maintain themselves Functions install and maintain themselves parasitical to cognition: to cognition:

functions install and maintain themselves thanks to and through agents' mental representations

but not as mental representations: i.e. without being known or at least

intended.

"Private stupidity/blindness, public benefit” (pseudo-Mandeville).


3. “Institutions” Construction We are and have to partially be blind also in "institutional" construction. For example, we reify, objectify as intrinsic to the object, act, role, certain powers and effects.

We cannot realize that those special "effects" of the institutional actions or objects (like a "signature" or the public declaration of a marriage), are the effects given to them by our assumptions and consequent and coherent behaviors (Searle); they are the effects of our (shared) beliefs and complementary actions. They are just self-fulfilling prophecies (expectations).

Our eyes and conducts "dress" the Emperor.


3. “Institutions” Construction We reify, objectify as intrinsic to the object, act, role, certain powers and effects.

Folk-psychology causal and ascription “mistakes” (Sperber)

are necessary,not only unavoidable, but useful for

social order,

like the idea of god, which works quite well.


4. Intrinsic Normative “Alienation”Norms Norms not only work thanks to our blindness and not fully understanding, but in a sense they "prescribe" so: ideally we have just to "obey", not to agree, to negotiate, to do for our convenience, or intending the "common good"; we are not supposed to understand the end of the norm. The aim of the norms is that I trust that they are for the "common good" and the group, the Polis (not for private desires and requests ), but not that I - by understanding what they are for - "cooperate" with the group/society. I have to "submit" to the (group) "authority": the decision has already been taken and was up to them not to me. By obeying I cooperate with a societal end that I do not understand/know and that I have to "pursue" even if I disagree about since (to me) the N is wrong or unfair.

> This was the stigmergic message of Socrates taking the poison.


4. Intrinsic Normative “Alienation”However, of course, differently from mere "social functions" there is a fundamental part part of the norms that must be mentally represented and understood. A norm works as a norm only if it is aimed at our "recognition" of it as a norm from an entitled power, etc. Not as a personal request, claim, imposition.

We have to acknowledge the norm and the authority, and to act for that reason, for a sense of duty, for submission.

So norms impose and presuppose a partial understanding of the social artifact; and a partial Goal-Adoption.

(BELOW)


5. Power Delegation and the Intrinsic Limits of

DemocracyThis "delegation" "delegation" of powers, of understanding, and of

decision, is in fact the the presupposition presupposition of the of the Polis, of the political powerPolis, of the political power; the structural limit of democracy and of the Polis as a "common good".

Representative democracy (the only real form of "democracy" not of demagogy) presupposes delegation, and non-understanding not only of technical solutions and non-local problems, but of possible compromises, political and social mediations.

We (possibly) have just to "trust" them for really representing and protecting "our" interests (but: "our" as group, class, lobby, or as community?!).

Also for that reason the societies are always objectively in favor of powerful, rich, and influencial people (even unintentionally).


6. Tutorial RelationsSociality and society need and build upon unavoidable

"tutorial" relations, founding not only public authority but several roles: parents, teachers, doctors, experts, advisors, ...

(i) "Interests" are not necessarily known, understood and thus pursued as our goals; we may do not understand what is good, better, for us, for our own goals (ignorance or incompetence or handicap or immaturity).

(ii) And it is possible that another person knows what would be better for us and takes care of our good (for several affective or ideal or role reasons); and we have to delegate and rely on him, trusting in his caring of our interest.

We call this relation "tutorial"; a fundamental, beautiful, unavoidable but very problematic and risky relation.

Also representative democracy is based on that impossibility of understanding and caring of our own interests, and on a supposed tutorial role of politicians and State.


Mind is not enough Mind is not enough Emergence, Self-

OrganizationFunctions vs. Cognitions

2


““COGNITIVIZING”COGNITIVIZING”Cooperation, Conflict, Power,Power, Social ‘Values’,

Commitments, Norms, Rights, Rights, Social Order,Social Order, Trust, , …

Pareto, Garfinkel, … the aim of founding the Social Pareto, Garfinkel, … the aim of founding the Social Sciences as Autonomous from PsychologySciences as Autonomous from Psychology


The “Cognitive Mediators”“Cognitive Mediators” of Social PhenomenaSocial phenomena are due to the agents’ behaviors, but…but…the agents’ behaviors are due the the mental mechanisms controlling and (re)producing them. (Castelfranchi, Conte, Miceli, Falcone,…)

For example:

• How the normnorm should work through the minds of the agents? How is it “represented”?

? Which are the proximate mechanisms underlying the normative behavior?


INDIVIDUAL MIND

COLLECTIVE STRUCTURES

& BEHAVIOURS

Bel --> G --> action

Mind is not enoughemergenceemergence & immergenceimmergencenot only knowledge, mutual beliefs, reasoning,shared goals and

deliberately constructed social structures and cooperation


“THE core theoretical problem of the whole social

science” (Hayek )

"This problem (the spontaneous emergence "This problem (the spontaneous emergence of an unintentional social order and of an unintentional social order and institutions) is in no way specific of institutions) is in no way specific of the economic science.... it doubtless is the economic science.... it doubtless is THE core theoretical problem of the whole THE core theoretical problem of the whole social sciencesocial science" " (von Hayek, Knowledge, Market, Planning)

the problem is not simply how a given equilibrium or coherence is achieved and some some stable stable orderorder emerges emerges

Is this emergence just an epi-phenomenon? Is this "order" only from the observer’s point of view?

To have a "social order" or an "institution", spontaneous emergence and equilibrium are not enough. They must be "functional".


Adam Smith’s "invisible hand"Adam Smith’s original formulation of “THE problem” is much deeper and clearer

The great question is how:

"(the individual) - that does neither, in "(the individual) - that does neither, in generalgeneral, intend to pursue, intend to pursue the the public interestpublic interest, , nor is aware nor is aware of the fact that he is of the fact that he is pursuing it,... pursuing it,... is conduced by an is conduced by an invisible hand to invisible hand to pursue an endpursue an end that is not that is not among his among his intentionsintentions"" (Smith, ). (Smith, ).

Hayek like Smith in acknowledging the teleological nature of the invisible hand and of spontaneous order, cannot avoid attributing to it

a (positive) value judgment, a a (positive) value judgment, a providential, benevolent, optimistic providential, benevolent, optimistic visionvision of this process of self- of this process of self-organization organization ((ideologismideologism).).


In the “Invisible Hand”:1) there are intentions and intentional behavior

2) some unintended and unaware (long term or complex) effect emerges from this behavior

3) but it is not just an effect, it is an end we “pursue”, i.e. its orients and controls -in some way- our behavior: we "necessarily operate for" that result (Smith).

- how is it possible that we pursuepursue something that is not an intention of ours; that the behavior of an intentional and planning agent be goal-oriented, finalistic (‘end’‘end’), without being intentional; - in which sense the unintentional effect of our behavior is an "endend”??


Theory of “Function”Theory of “Function”This problem appeared in other social sciences as the problem of the notion of "functions" (social and biological) impinging on the behavior of anticipatory and intentional agents, and of their relations with their "intentions".


Social Functions Social Functions and CognitionCognitionWhile Social NormsSocial Norms emergence and functioning require also a (partial) "cognitive cognitive emergenceemergence", Social FunctionsSocial Functions require an extra-cognitiveextra-cognitive emergence and working

For a Social Norm to work as a Social Norm and be fully effective, agents should recognize and treat it as a Social Norm. On the contrary the effectiveness of a Social Function is independent of agents' understandingindependent of agents' understanding of this function of their own behavior:


The problem: Emergence and Functions should not be

what the observer likes or notices, (“just in the eye of the

beholder”)but should be indeed observer-independent, based on self-organizing and self-reproducing phenomena, >>> "positive”, “good” can just consists in this.

>> Two kinds of finalistic notionsfinalistic notions: - evolutionary finalities,

adaptive goals; and - mental ends (motives,

purposes, intentions).


Intentional Intentional behavior Vs. functional functional behavior

Finalistic systems:

There are twotwo basic types of system having a finalistic (teleonomic) behaviour:

Goal-oriented systems - (Mc Farland, 1983),

Goal-governed systems

a specific type of Goal-oriented system based on representations representations that anticipate the results


Goals Goals vs. “Functions” “Functions”


What distinguishes function from non function is not not that the unintentional (collective) effect is good but that it is self-organizing and self-producing by means of positive feedback, that is, by reinforcing, selecting, and reproducing the behaviour that generated it:

unintended effects that unintended effects that selectselect their own their own causescauses.

>> functionality must be kept distinct from goodness (and that is from the subjective goals of the agents), >> good and bad functions (exactly like unintended good and bad effects) are on the same plane: both may be self-organizing.


Goals Goals vs. “Functions” “Functions”

An example:

The ADOPTION of a NORMADOPTION of a NORM without the UNDERSTANDINGUNDERSTANDING (sharing) of its END


A bit more onNormative Alienation

3


The a-technical, non-rational nature of the deontic “ought”deontic “ought”

The “alienated” “alienated” nature of norm adoption

&


The a-technical, non-rational nature of the deontic “ought”the deontic “ought”

Both the ‘ideal’ and the ‘sub-ideal’ (for avoiding sanctions) obedience share a fundamental core, crucial for the real nature of the deontic ‘norm’, ‘ought’.

A core that differentiate the mind of the normative ‘subject’ S from the mind of the ‘issuer’ or legislator.S is NOT supposed to (have to) adopt the N (to ‘adhere’ to the imperative) because s/he understands or agrees about its function, aim.On the contrary, S is supposed to have to to obey even if s/he does not understand the obey even if s/he does not understand the meaning of the N, or disagrees about it.meaning of the N, or disagrees about it.A normative education is precisely an education to obey in any case, and even to not wondering and worry about the validity of the N. (That’s why Socrates takes the poison)


Citizens like childrens

We are not supposed to ‘cooperate’ but to ‘obey’!


The a-technical, non-rational nature of the deontic “ought”

In a sense the deontic “ought” “have to” is a de-technicalized “ought”: no longer a necessary means for.... means for.... something that you have to want, to chose.you have to want, to chose. “If you like/want to ..., you should, have to.....” (technical ought)“You have to” for what? why? (deontic ought) In the mind of the “issuer” the N is supposed to be a means, a solution for some problem, a way of achieving a higher goal.


The “alienated” “alienated” nature of norm adoption and on institution and

social powers

We do not intend the ends of our intentional

behavior!


SOME AI CONSIDERATIONSin an AI Perspective

Social Simulation, ICT-based HYBRID SOCIETY

Might it help?


““Socio-(Cognitive-)Technical Systems”Socio-(Cognitive-)Technical Systems”What we are unavoidably building with computer networks, AI, and Ag technologies are Socio-Cognitive-Technical Systems:

Socio-Technical System in fact means that any new technology implies/requires/introduces not only new skills and competences, but new expectations, goals, beliefs; new "scripts", with their roles, norms; new form of interaction and conventions among the social actors. So we have to specify the "cognitive" and interactive side of the new system.


““Socio-Cognitive-Technical Systems”Socio-Cognitive-Technical Systems”BUT….. this new complex Socio-Technical (and mental) System cannot be just planned cannot be just planned and designedand designed. It is dynamically emerging and self-organizing: it is a spontaneous Social Order spontaneous Social Order (von Hayek); a dynamic equilibrium not necessarily "good" for the goals of the actors.

What we need is not just a top-down organization and control.


““Socio-Cognitive-Technical Systems”Socio-Cognitive-Technical Systems”in orderto support and mediate human interaction and organization and to emulate them in efficient open MAS systems,

we have to (partially) "understand" and to reproduce features of human social mind (like commitments, norms, mind reading, power, trust, "institutional" effects, ...) and of social macro-phenomena.

In particular we have to model "immergence" "immergence" and "cognitive emergence"cognitive emergence"; the mental "mediators" of societal phenomena and the partial understanding and awareness in the actors


““Socio-Cognitive-Technical Systems”Socio-Cognitive-Technical Systems”We have to "understand" and to reproduce also

hhow humans do socially construct ow humans do socially construct something something without understanding without understanding itit! ! How is it possible that intentional agents do not intend the functions of their collective behavior? Which the relationship between emergent functions and intended goals?

CAN WE SUPPORT HUMAN ORGANIZATIONS & BUILD EFFECTIVE SOCIAL SYSTEMS WITHOUT UNDERSTANDING AND GOVERNING THAT!?


Only MAS can solve this theoretical and technical problems; by changing the Social Sciences methods, data, and modeling tools. By modeling Emergence Emergence but also Immergence Immergence and Cognitive EmergenceCognitive Emergence, and different levels and kinds of order; how minds become coordination artifacts; different kinds and levels of cooperation. And in particular the relation between mental representations and learning and self-organizing autonomous systems.

Can we Program with the "Invisible Hand"? Will the "Invisible Hand" - governing human society - be implemented in the emergent intelligence of Self-organizing open Self-organizing open MA systemsMA systems?


MAS will play several crucial roles:

- Experimental methods and platforms for the Social and political sciences;- New conceptual, theoretical and modeling tools;- Infrastructure for Self-organizing but monitored and governed organizations: MA implementation, Ag mediation and support;- Participation, transparence, information; mediamorphosis of knowledge institutions;- Simulation for understanding, predicting and deciding about real social dynamics;- MA coordination/interpretation of feedbacks from intelligent sensors, stakeholders, people, ... to authorities and control institutions.

Let me stress just one:

Artificial Social ImaginationImagination


Concluding Concluding Remarks Remarks


Self-Organization Self-Organization = = Out of MindsOut of MindsSociety works thanks our partial intelligence.

We cooperate and jointly act not just in spite of but thanks to thanks to the fact that we do not not (fully) understand and intend what we are jointly doing.

And even if we disagree.

Obviously there are also very dysfunctional and undesirable results of our stupidity/blindness.

For example, since we do not understand or forget, and thus do not learn from the worse tragedies of our History, that's why we repeat them as farces.


• Reconciling EmergenceEmergence and CognitionCognition

1) Cognition as a level of emergence:

- from objective to subjective- from implicit to explicit ex. dependence

2) Beyond cognition: emergent unaware, functional social phenomena

(ex. cooperation, problem solving) among cognitive/intentional agents

the new synthesisnew synthesis


“AlienationAlienation”:We are dominated by our own

‘delegated’ (emergent) (social and intellectual) powers, we are not aware of;

we do not realize and we do not decide/intend to create such collective or such institutional powers impinging on us.


Can We Overcome our AlienationAlienation?Will the Leviathan become

a giant connected and informed connected and informed community of agentscommunity of agents, managing their collective Power?1. I’m skeptical about that (also for cognitive reasons)2. I worry about possible net-Demagogy


DemocracyMark Twain is brilliantly right

"If voting made any difference they wouldn't let us do it."

But… the problem is much harder; it is not just a complot, is that we vote in a self-defeating way, and, in general, our collective stupidity.

Might political "education" and education to "commons" be enough, and solve the problem?

Given the immediate local perception of the conflicting interests and competition and the blindness to common interests among different countries and poor classes and ethnic groups, and affiliation and identity feelings, conformism, and in-group vs. out-group psych, ... I have some doubt.

In a couple of centuries they will see.


ENDENDThank you for your

attention!

And sorry for my “English”


I like to thank our research group in Cognitive Science at ISTC: the ‘‘GOAL GOAL group’group’Maria MiceliRino Falcone(Emiliano Lorini)Fabio PaglieriGiovanni PezzuloLuca Tummolini

A part from Rosaria Conte (Social Simulation LABSS Group)

http://www.istc.cnr.it/group/goal


When you find yourself on the side of the majority, it is time

to stop and reconsider Mark Twain


7. How God(s) WorksEven (the idea of) God couldn't work, be so effective

in human societies, without this collective (useful) self-deception: we "put into practice" with our blind actions and interpretations of the effects and causes.


Hobbes Hobbes or DurkheimDurkheim?“Homo Homini Lupus”

or“Cohesion” & “Solidarity” ?

x


The Same Foundation The Same Foundation !! Of the TWO faces of Sociality> > From Interference & Dependence to

COOp & EXCHANGE> From Interference & Dependence to

COMPETITION & FIGHT


AgentsINTERFERENCE in a common world

ror

q

q p

q

andp objectiveDEPENDENCEnetwork

EXCHANGE& COOPrelations

Levels ofemergence...


AgentsINTERFERENCE in a common world

ror

q

q p

q


COMPETITION& FIGHTrelations



ror

q

q p

q


COMPETITION & WARon the top ofExchange & Coop

Levels ofEmergence ... CONFLICTS

COOPERATION


ror

q

q p

q


EXCHANGE& COOPon the top ofCompetition& Fight

Levels ofEmergence ... ALLIANCES

CONFLICTS


Only MAS can fully deal with this problemOnly MAS can fully deal with this problem>> Up & Down>> Up & Down


Why alsoKako-functions?Kako-functions?

How is it possible?


Why also kako-functionskako-functions?

- thethe mechanism that install a bad mechanism that install a bad function can be exactly the same function can be exactly the same installing a good oneinstalling a good one

- to definitely separate a functional view of behavior and society from any teleological, providential view (functions can be very bad and persist although bad)

- kako-functions cannot be explained in a strictly behavioristic framework of reinforcement learning: the result of the behavior can be disagreeable or useless, but the behavior will be "reinforced", consolidated and reproduced.


Unexpected evil effects exist, or evil effects combined with good individual intentions (Boudon, 1977) in which

the intended good effects reproduced the intended good effects reproduced in spite ofin spite of the negative the negative

consequencesconsequences. This is true,

- both in the case in which the evil effects are not perceived or are not attributed correctly,

- and in the case in which they are perceived(in the second case the good effects must be subjectively more important and in any case preferred (for instance, be closer in time), or else are more conditioning/reinforcing than the evil effects)


But there are also harmful effects capable of self-reproductionharmful effects capable of self-reproduction (through the action) precisely because of their because of their negative naturenegative nature (Castelfranchi, 1997; 1998b; 1998d).

a long line of automobiles and the slowing down due to the simple individual intention of rapidly glancing at an accident that has occurred in the other lane


The notion of ‘function' as an effect selecting and an effect selecting and

reproducingreproducing its own its own

causecause

How is it possible for a system that acts intentionally on the basis of an evaluation of the effects vis-à-vis its own goals, to reproduce bad habits precisely as a result of their bad effects?

And even more crucially - if a behaviour is instead reproduced thanks to its good effects with respect to the (individual or collective) goals of the agent who reproduces them by acting intentionally, then there is no room for the "functions".


It is necessary to have complex reinforcement learning forms not merely based on classifiers, rules, associations, motor sequences, etc. but operating on the cognitive operating on the cognitive representationsrepresentations governing the action, that is, on beliefs beliefs and goalsgoals.

In this view "the consequences of the action, which may be more or less consciously anticipated, nevertheless modify the probability of the action being repeated the next time in similar stimulus conditions " (Macy, 1998). More exactly:

the functions are simply effects of behaviour the functions are simply effects of behaviour which go which go beyond thebeyond the intended effectsintended effects but but which can successfully be reproduced because which can successfully be reproduced because they reinforce the agent's beliefs and goals they reinforce the agent's beliefs and goals that give rise to this behaviour.that give rise to this behaviour.


How Social Functions are implemented through

cognitive representations

The basic model


There are two Cognitive "reinforcement" principles:

1. Belief Reinforcement: two different mechanisms can be postulated:

association (accessibility) : the association between the belief and that context or scenario is strengthened: the believe will have more probability to be retrieved next time in similar situations; it will be more activated, more available and accessible (accessibility bias);

confirmation (reliability) :some of the action's effects are perceived by the agent (even if not necessarily understood and causally connected to its actions) and they confirm the beliefs supporting the action: they give new evidence for that belief, increase its "credibility", and reliability: they augment its “truth” or the subjective probability of the event.


2. Goal Reinforcementtwo different mechanisms can be postulated (analogous to the beliefs reinforcement mechanisms):

association (accessibility) : the success of the chosen goal, plan, action is memorized in the sense that the association between the goal-plan and that problematic context or scenario is strengthened: the goal/plan (solution) will have more probability to be retrieved next time in similar situations; it will be more activated, more available and accessible;

confirmation (reliability) :the success of the chosen goal, plan, action is memorized; it increments a "successfulness index" relative to that choice; or better some meta-cognitive evaluation of the value of the action. This memorized behavioral choice is "confirmed": next time the probability to choose the same way (goal, plan, strategy, action) will be greater: it will be more preferable and reliable (we will trust more it).

The reinforcement of both the belief and the goal/plan

will determine a reinforcement of that behavior


An example: dirty and clean streetsA social (kako)function based on social conformity and imitation. The agent assumes (B1) that this is a bad behavior or even a forbidden one; he assumes (B2) that a lot of other people behave this way; that (B3) this can be quite practical and easy sometimes; he assumes that (B4) his contribution to the garbage is quite marginal and small (that its true). He has the goal (G1) to do as others do and until others do so (Bicchieri, 1989); or at least, to do as others do and until others do so if this is useful and practical for his goals. Goal G1 on the bases of beliefs B2, B3, B4 will generate a goal (G2) to leave small garbage in the street, which overcomes the possible goal (G3) -based on B1- of not dirtying the city. Now the result of such a behavior is that streets are dirtier; this is perceived and then it will confirm the supporting beliefs (B2, B4) and the goal G2.

Everybody reinforces the behavior of the others. The global effect is not wanted and intended by anybody;

the reinforcement effect is also unattended and unintended.

The behavior is (reciprocally) reinforced by its effects. These effects are self-maintaining and reproducing through the reinforcement of their own causes. This passes through the mind of the agents (their beliefs and goals) but not through their consciousness and intention.

It is quite interesting to observe that exactly the same kind of beliefs, and an identical goal (G1) can generate in this case an eu-function: to maintain the city clean.


Interpersonal and collective level: vicious circlesvicious circles

Hubbub in a restaurant or at a party

(The example given is merely -on a small scale- the model followed by the arms race).

Hostility leads to hostility


Institutional level: vicious circlesvicious circles

Prisons <==><==> Delinquency


INDIVIDUAL MIND

COLLECTIVE STRUCTURES

& BEHAVIOURS

Bel --> G --> action

2- Mind is not enoughemergenceemergence

not only knowledge, mutual beliefs, reasoning and

deliberately constructed social structures and cooperation


Programming (with) ‘the Invisible Hand’??The problem of Emergent Computation and DAI/MAS

• Emergent Computation (Forrest, 90):

» The idea that interactions among simple deterministic elements can produce interesting and complex global behaviors is well accepted in sciences. However, the field of computing is oriented towards building systems that accomplish specific tasks, and emergent properties of complex systems are inherently difficult to predict and control.

» It is not obvious how architectures that have many interactions with often unpredictable and self-organising effects can be used effectively.

» The premise of EC is that interesting and useful computational systems can be constructed by exploiting interactions among agents.



– The important point is that the explicit instructions are at different (and lower) level than the phenomena of interest. There is a tension between low-level explicit computations and direct programming, and the patterns of their interaction.

• Indirect programming: – implementing computations indirectly as emergent patterns

• Forrest: connectionist models, cellular automata, biological and ALife models

» also higher level components: complex AI agents, cognitive agents



• Central themes of EC include (Todd 93):» self-organisation, with no central authority to control the overall flow of

computation;» collective phenomena emerging from the interactions of locally-communicating

autonomous agents;» global cooperation among agents, to solve common a common goal or share a

common resource, being balanced against competition between them to create a more efficient overall system;

» learning and adaptation (and autonomous problem solving and negotiation) replacing direct programming for building working systems;

» dynamic system behavior taking precedence over traditional AI static data structures.

• Typical DAI, MAS issues– Also higher level components: complex AI agents, cognitive agents– i.e the problems of human society: functions and ‘the invisible hand’

• spontaneous emergent of order, beneficial self-organisation, the impossibility of planning but also harmful self-organising behavior


81

BUT.... wasn’t this the “horrible” and unpronounceable “planning” of the so-called ‘socialist’ countries? WHY big word corporations can program interventions and (planetary) policies for longer periods than any government, and for much larger capitals than for a lot of nations? While those nations/governments cannot even speak of “plans”, otherwise they have an anachronistic, anti-liberal, inefficient view?

political issues and challenges


82

Were perhaps the main limits of the “economic plan” of “socialism” also cognitive cognitive limitslimits?? There was no collective “brain” really apt to calculate, predict, adjust, those very complex dynamics, and take into account in real-time the feedbacks and the signals from the social reality (on the ground)?

political issues and challenges


This principle is unavoidable for both rationality, planning, and cognitive learning.

To account for functions, we should admit somesome mechanism that reproduces the mechanism that reproduces the intentional action thanks to (some of) its effectsintentional action thanks to (some of) its effects, , butbut

bypassing agent understanding and planning these effects (that can even be good for its goals and reproduced for that).


MAIN PROBLEMSMAIN PROBLEMS

• If a behavior is reproduced thanks to its good effects, that are good relatively to the goals of the agent (individual or collective) who reproduces them by acting intentionally, there is no room for "functions” (Elster).

If the agent appreciates the goodness of these effects and the action is replied in order to reproduce these effects, they are simply "intended".

• How is it possible that a system which act intentionally and on the basis of the evaluation of the effects relative to its internal goals reproduces bad habits thanks to their bad effects?


>> ?? a behavioristic reinforcement layer (van Parijs)

together with>> a deliberative layer (controlled by beliefs and goals) ???

the deliberative layer accounting for intentional actions and effects, the behavioristic layer (exploiting conditioned or unconditioned reflexes) accounting for merely "functional" behaviors??

Are “functions” and “roles” just impinging on ‘habitus’ ???(Bourdieu), while

intentions would just be for personal purposes??

Our problem is indeed that: intentional actions intentional actions have functionsfunctions!

Goals and beliefs of the agents have functions.


Contrariamente a quanto sostiene la teoria della “estinzione dello stato” e del governo delle cose ma non piu’ degli uomini (coscienti cooperatori e gestori delle dinamiche sociali), non e’ possibile una eliminazione totale di questo rapporto cooperativo alienato; del cooperare (conformandosi) al bene collettivo, senza capirlo e quindi senza davvero intenderlo. Non e’ possibile un cittadino pienamente ‘consapevole’ e cooperante con le scelte pubbliche, cui quindi nulla viene davvero ‘imposto’, neppure le ‘imposte’. Egli resta comunque almeno in parte un ‘suddito’. Non e’ possibile per limiti cognitivi e culturali dei singoli e dei gruppi; non e’ possibile per la facile prevalenza mentale di interessi personali, di categoria, locali, a breve termine, sugli interessi collettivi ed a lungo termine, e con i loro necessari compromessi e sacrifici; non e’ possibile per problemi di tempestivita’ e regolarita’ del coordinamento di innumerevoli condotte distribuite.E’ comunque necessaria una quota di ‘delega’ e di non-comprensione dei perche’ e dei fini; possibilmente basata su una ‘fiducia’ verso i governanti o i potenti, e non su un timore del loro potere e delle loro sanzioni.

The “alienated” nature of norm adoption


Il fatto che un ‘suddito’ della N non possa/debba capire il senso tecnico della N (che questo sia per lui e per la sua obbedienza ‘irrilevante’ e che il suo parere tecnico sia ‘irrilevante’) non e’ certo garanzia del fatto che N non sia in realta’ rivolta a fini diversi dal “bene comune”. Io S mi fido o comunque subisco, non posso capirlo. Servira’ la N in realta’ piu’ a fini di classe o di parte? Servira’ agli interessi dei ‘legislatori’?

Vi e’ “alienazione”nel senso che il soggetto aliena le proprie capacita’ di decisione e problem-solving, ‘delega’ ad altri detta capacita’, potere e soluzione; ed inoltre non e’ in grado di comprendere pienamente cio’ ed agisce non riconoscendo i propri poteri alienati e non riappropriandosene.

Y –a cio’ delegato – ha trovato la soluzione del problema collettivo, ma io non devo capirla, riconoscerla, adottarla in quanto tale. Devo farlo ‘al buio’.

The “alienated” nature of norm adoption


Agent-based Modeling & Social Simulation will be crucial for the solution of one of the most hard theoreticaltheoretical problems of economic and social sciences:

the spontaneous organization of a the spontaneous organization of a “dynamic social order” that cannot be “dynamic social order” that cannot be planned, but emerges out of planned, but emerges out of intentional planning agentsintentional planning agents guided by guided by their own choices.their own choices.

This is the problem that Hayek assumes to be the real reason for the existence of the Social Sciences.


““Joint Intention”, Collective Joint Intention”, Collective Action, …Action, …

A naïve view

4


Self-Organization = Out of MindsSelf-Organization = Out of MindsIn sum, it was quite naive the hold position

trying to represent joint intentions and collaboration in terms of "shared" plan;

(i) we work in one and the same plan - intentionally and mutually knowingly - without necessarily specifying in each mind those parts of the plan in charge of the others; our mental representations are complementary and partially blindpartially blind.


Goal

action

subGoalsubGoal

PLAN

Des.

action action

Goal

action

subGoal

?

Open-DelegationOpen-Delegation: • Delegation of meta-tasks (searching,

planning, deciding, monitoring, ...)• Intelligence, autonomy (agent modelling)

Levels ofLevels of Delegation Delegation


Goal

action actionaction

subGoalsubGoal

PLAN

Des.

Goal-DelegationGoal-DelegationVs Goal-AdoptionGoal-Adoption

• Over- HelpOver- Help• Critical-HelpCritical-Help

Levels of collaboration


non shared plannon shared plan Goal

action

subGoal subGoal

action action

subGoal

GoalPLAN

action

subGoal

PLAN Goal

action

subGoal subGoal

action

Unaware CooperationUnaware Cooperation

•Reconciling EmergenceEmergence and CognitionCognition


Self-Organization Self-Organization = = Out of MindsOut of MindsIn sum, it was quite naive the hold position trying to

represent joint intentions and collaboration in terms of "shared" plan;

(ii) We even work in one and the same plan without any awareness of that; only the “boss”, the organizer or buyer of our work knows about our “collaboration” with other guys. The MA plan is in his mind not in my mind.

_______________Even more naive would be the attempt to build

society on a mutual or at least shared understanding of our collective functional outcomes ("spontaneous order") and of (mentally mediated) coordination artifacts, like roles, norms, conventions, money, .. .


The “Cognitive Mediators”“Cognitive Mediators” of Social PhenomenaSocial and cultural phenomena cannot be deeply accounted for without explaining how they work through the individual agents’ minds (mental “counterparts” or “mediators”).

This requires a richer cognitive model (architecture) for “Agents,” moving from formal and computational AI and ALife models, closer to those developed in psychology, cognitive science, and in cognitive approaches in economics, sociology, organization studies.

“The most important fact concerning human interactions is that these events are psychologically represented in each of the participants” (Kurt Lewin, 1935)


Agents in a common world (INTERFERENCE)

ror

q

q p

q

andp

G

objectiveDEPENDENCEnetwork

Cognitive Cognitive emergenceemergence:awareness



However, Mind is not enoughMind is not enough!!

the “individualistic + cognitive” approach is not sufficient for the social theory and processes (even when modeling joint and collective attitudes and actions).

The social actors do social actors do notnot understand, understand, negotiate, and plan negotiate, and plan for all their collective behavior and cooperative activity.


Trusting Trusting & Delegating & Delegating without without

UnderstandingUnderstanding(or Agreeing)(or Agreeing)


Norms exploit and count on a special process/kind of Goal-special process/kind of Goal-

AdoptionAdoption

• First, they count on Goal-Adhesion: that is, on the recognition by the addressee of the will of the issuer, and on an adoption due also to this: I adopt your goal also because I know that you want so. ““Obedience”Obedience” in general is a sub-kind of “Adhesion”, and norm obedience is a kind of obedience.

• Second, it is a non “personal”, individual request, but it is a generalized request, and should be understood as such and used as such.

• Third, it should be motivated by the sense and respect of the authority and values; not by external rewards.

But, more important….


The a-technical, non-rational nature of the deontic “ought”

Two kinds of education:

Understanding & responsibilization: “Hai visto che succede....” “Ti rendi conto di cosa hai fatto....?” “Questo non si fa’; senno’ vedi che guaio?”.

No understanding/explanation, no sharing:(iia)“Non si fa e basta!” “Perche No; perche’ lo dico io!” “Perche’ e’ probito!” (iib) “Senno’ son botte”.

The norm issuers are supposed to have in mind the aim and instrumental function of the norm


Not only the MAS supported social order cannot just be top-downcannot just be top-down, but it must allow autonomy and flexibility, and true delegations and commitments, and "conventions" and "norms" (top-down & bottom-up) with their intrinsic possible “violation”.

We need cooperation/support by “autonomous”“autonomous”, “pro-active”“pro-active”, really , really helping helping Agents.

Not stupid executors.

Collective Unintelligence: The Ground of Society

Documents

Transcript of Collective Unintelligence: The Ground of Society