SITO: Social Impact Theory based Optimizer –Recent Challenges

Martin Macas, Lenka Lhotska Czech Technical University in Prague

Faculty of Electrical Engineering Department of Cybernetics

Technicka 2, Prague 6, 166 27, Czech Republic Phone: +420 224 357 666, Fax: +420 224 923 677

email: lhotska@fel.cvut.cz

ABSTRACT: The Social Impact Theory based Optimizer (SITO) is a novel population-based binary optimization meta-heuristic, inspired by computer simulation of Dynamic Theory of Social Impact. Its first version has been developed in 2006. Recently, we can summarize some of most important aspects of SITO, including some ideas for future work.

KEYWORDS: Social psychology, optimization, social impact theory, population based optimization.

INTRODUCTION

This paper follows up with connections between Social Impact Theory [1] and optimization introduced in [2]. Social Impact Theory based Optimizer [3] is binary optimization method inspired by social psychology. Moreover, evolutionary computation, cellular automata or artificial life can be also understood to be precursors of the method. The algorithm is based on a population of simple cells. These cells are inhabited by individuals representing candidate solutions - binary vectors. Each such binary vector is a counterpart to set of binary attitudes held by real people. These attitudes are changed over time according to external influences and internal processes. Each individual in SITO has its strength – ability to influence (persuade or support) the other individuals. The strength is derived from the distribution of fitness values in the society – the better the individual’s fitness, the stronger the individual. At each iteration, each individual for each dimension of the candidate solution computes total supportive and total persuasive impact representing the influences of the individual’s neighbourhood. If the persuasive impact is the predominant one, the particular binary number (attitude) is changed with a predefined probability. Thus, something like selection pressure arises and the population starts to converge and search the binary space.

S.I.T. OPTIMIZER

The optimization problem considered in this paper is to find a binary vector ),...,( 1 Diii xxx =

r, that extremizes a fitness

function . Consider a population of individuals. An individual has associated a binary vector

representing candidate solution of a D-dimensional binary optimization problem, which can be evaluated using the fitness function. Further, each individual has its position in the population topology, which makes it possible to define distance between two individuals and j, . Concretely here, will be the Euclidean distance

between positions of individual and individual

)( ixf r i),...,( 1 D

iii xxx =r

i jid , jid ,

i j . At the beginning, the binary values , where

are randomly initialized with uniform probability. At each iteration, all vectors are

evaluated ( is computed for all i ) and strength factors

pix

)...1(),...1( DpNi ==)( ixf r )/())(( minmaxmin fffxfq ii −−=

r are

determined. This implies that . Further, each individual inverts the bit if >∈< 1,0iq i pix

∑∑∈∈

++>Sk

Spiki

pk

OjOji

p NqdqNdqj

1/)/)/((/)/( 22,

2, δ , (1)

where is the set of indices O j of individuals for which (opponents) and S is the set of indices k of

individuals for which (supporters) and

pi

pj xx ≠

pi

pk xx = ik ≠ . The term on the left-hand side of the inequality (1) is the

total supportive impact and the term on the right-hand side is the total persuasive impact. The term represents

the self-influence.

2/δpiq

δ is a constant self-distance parameter, which determines the relative importance of the self. The inversion of a particular bit can occur even if the persuasive impact is less than the supportive one. On the

other hand, if the persuasive impact predominates, the change may not occur. It is enabled by an additional probabilistic parameter κ which represents the probability of spontaneous change. Thus, if persuasive impact is greater, the change takes place with probability κ−1 , else, the change takes place with probabilityκ . It can improve the explorative capability and prevent loss of diversity.

It is obvious that the described algorithm has many adjustable parameters that more or less influence the optimization process. Moreover, there is a wide variety of modifications of the simple original version and we try to describe here some of most important aspects and approaches that could lead to some improvements.

NEIGHBORHOOD

At the original version of the algorithm, each individual takes into account the influence from all individuals in the society, however the influence is weighted using , which introduces the concept of immediacy and corresponds to the fact that two individuals living far between each other has much smaller influence to each other than two neighbours. The fact that the influence decreases with squared distance corresponds to empirical data

2,/1 jid

[4]. The steeply decreasing function (Chyba! Nenalezen zdroj odkazů.) leads to highly suppressed influence of distant individuals. The influence of these individuals can be neglected and thus the set of opponents O and the set of supporters S can be restricted to a neighbourhood with predefined radius. There can be two basic types of neighbourhood – the von Neumann neighbourhood and the Moore neighbourhood. The radius of the neighbourhood can significantly affect the result of optimization and the computational requirements. Moreover, it must be pointed out, that the distance exponent needs not to be 2. An exponent of 0 means, that all individuals are uniformly distant to each other and the influence is not affected by the distance. A higher exponent effectively limits influence to the closest neighbours.

2,/1 jid

Figure 1: (a) The function weighting the influence between two individuals i and j at the distance from each other (b) examples of the von Neumann and Moore neighbourhood respectively (radius=1).

2,/1 jid jid ,

STOCHASTICITY

It has been already remarked above that the randomness has been introduced into the algorithm, which can help to prevent loss of diversity and improve exploration. The random element has been added using the probability of spontaneous changeκ . This influence of this parameter can be seen from Figure 2: and Figure 3:. Figure 2: presents the time evolution of the maximum fitness value. A feature selection problem corresponds to these figures and the curves are averaged over 30 runs. The best final result (after 200 iterations) is reached for 15.0=κ . Figure 3: depicts the time evolution of the diversity. The diversity has been evaluated using the following measure: diversity measure = number of different values of fitness/population size. The Figures show that the stochastic component has a great influence on the result of optimization. If the parameter κ is too small, the diversity decreases slowly from the beginning, there is not enough randomness and the optimization process gets stuck in a local optima. The high value of κ prevents the loss of diversity, however, the stochasticity is too high and the algorithm has problems with fitness improvement. It seems like there is a critical moment in the time. After this point, the diversity starts to decrease with linear-like trend. The temporal position of this critical is determined by the parameter κ (see Figure 2).

Figure 2: The time progress of the maximum fitness value averaged over 10 runs.

Figure 3: The time progress of the diversity measure averaged over 10 runs.

However, there could be some other ways, how to adding a noise. An example is the temperature. It can be defined as a random variable drawn for each individual at each time step from a uniform distribution with zero mean. This variable is added to the persuasive impact. In original simulation [5], the moderate temperature led to a stable equilibrium with stable and even increased degree of order, but the process had a great deal of individual change. The

amount of individual changes could be an advantage in our processes of optimization. The examination of the temperature parameter belongs to the future work.

SELF-CONFIDENCE

Another phenomenon, which should be examined, is the self-confidence. This factor is introduced using the self-distance parameter in the update equation (1). It determines a relative importance of the self. The higher the self distance parameter δ , the smaller the confidence in one’s own attitude. Actually, this factor adds a bias to the total supportive impact and it could be reasonable to set δ inversely proportional to time of keeping a particular opinion. However, in the preliminary experiment, the δ has been set to 1.

SITO CHALLENGES

PRACTICAL APPLICATION OF SITO

The basic SITO algorithm has been already tested and compared to other methods on several problems. Most of them were artificial benchmarks. The SITO is comparable (not statistically better or worse than the others) to the basic version of genetic algorithm and to the binary particle swarm optimizer.

However, just one practical application has been implemented – the feature subset selection [6]. The results have shown purposeful and successful application of the SITO in the field of dimensionality reduction for pattern classification. The SITO has been comparable to binary particle swarm algorithm. However, it could be good to find an application where the benefit of adopting SITO is clear.

THEORETICAL ASPECTS

Much work on nature inspired and population based algorithms aimed on experimental studies deducing some characteristics of a particular method from experimental observations. It is however important to be able to confront the experimental results with a theory. The lack of a well established theory is main problem of majority of nature-inspired, population based and stochastic optimization techniques. Most studies simply take some metaphors from nature and apply it directly. The theory, however, could help us to inspect and predict an influence of method’s parameters and thus find a proper parameter setting. Formal proof of method’s convergence can also help us to present and propagate the SITO. Thus, the need for a good theory is not just on account of scientific manners, nothing is more practical than a good theory.

One of very promising theoretical approaches could be taken from the attempts for theoretical description of social simulation using statistical physics [9], [10]. Statistical physics, particularly the statistical mechanics allows to predict macro-level phenomena resulting from micro-level rules. However, these approaches sometimes need to take some theoretical assumptions (e.g. infinite number of individuals). Another interesting theoretical point of view is an examination of complexity, where many findings from the field of cellular automata can be used (the SITO algorithm shares most of its properties with cellular automata).

MODIFICATIONS

The SITO algorithm is a novel and immature technique. It has been developed and its first implementation was performed during 2006/2007. The early form of the method is strongly associated with the social simulation of Latané social impact theory [7]. However, there is no special reason why to keep the original form. Thus, many options for changing the original paradigm appear. The modifications can be connected with different aspects of the algorithm, no matter if it will be a modification of the topological structure of the society, different type of the update rule (1) or different fitness-strength mapping. Another important challenge is a transfer of the basic model into the continuous optimization field.

FURTHER CONSIDERATION

The SITO algorithm is a population based, stochastic binary optimization algorithm. It is therefore apparent, that it will share many properties with other optimization techniques of such type. It is important to consider a similarity with a family of evolutionary computation techniques. The main operators involved in these techniques are selection, crossover and mutation. Is it possible to find any analogy to these operators in SITO? It is obvious that in the original version, the individuals are not selected and all of them survive into the next generation. However, each individual has its own strength, which weights its relative importance. This can be understood as a special type of selection. If we understand the crossover of two individual in evolutionary techniques as a form of influential process, this influence is reciprocal and each individual influence the other one at the same rate (simple change of chromosome substring). This is not true in SITO, where the fitter individual has higher impact on the worse one. Finally, the mutation is the only operator, which takes place in the same manner for both the evolutionary techniques and the SITO algorithm. In this point of view, the parameterκ has the same function as the mutation rate. One interesting aspect of SITO is that the society of individuals is spatially distributed, this case can be observed in spatial co-evolution [8], where the populations are spatially embedded on a predefined lattice and selection is performed locally in space – each individual competes for survival against the surrounding individuals in its neighbourhood.

Another population based techniques, which should be mentioned here is the binary particle swarm optimization (bPSO). In contrast to the continuous particle swarm optimization inspired by movement of swarms of creatures, the bPSO has quite different real-world analogy - a model of binary decision, where individuals are represented by the decision vector and the goal is to find an optimal binary pattern of some choices. The decision vector can be understood as the set of binary attitudes and the main mechanism propelling the optimization is the social impact of the individuals in the society.

CONCLUSION

This paper propounded the questions concerning the Social Impact Theory based Optimization. It described the most critic points of the method and some open issues which should be considered in the future. Although some tests have been already performed, the main topic which should be studied as the first is an experimental proof of a practical value of the method. It should be done by experimental testing on wider range of benchmark problems. The second future plan is to find a formal theory describing the whole process. ACKNOWLEGEMENT: The research was supported by the research program No. MSM6840770012 Transdisciplinary Research in the Area of Biomedical Engineering II of the CTU in Prague, sponsored by the Ministry of Education, Youth and Sports of the Czech Republic and by the FP6-IST project No. 13569 NiSIS (Nature-inspired Smart Information Systems).

REFERENCES

[1] Latané, B., The Psychology of Social Impact. American Psychologist, 1981. 36(4): p. 343-356.

[2] Macaš, M. and Lhotská, L., 2006, „Social Impact and Optimization”, 2nd European Symposium on Nature-inspired Smart Information Systems, www.nisis.de.

[3] Macaš, M. and Lhotská, L., 2007, „Social Impact Theory based Optimizer”, Advances in Artificial: 9th European Conference on Artificial Life, Heidelberg: Springer, pp. 635-644.

[4] Latané, B., Liu, J. H., Nowak, A., Bonevento, M. and Zheng, L., 1995, “Distance matters: Phsical space and social impact”, Personality and Social Psychology Bulletin, 21, pp. 795-805.

[5] Nowak, A. and Latané, B., 1993, “Simulating the Emergence of Social Order from Individual Behaviour”, Simulating Societies: The Computer Simulation of Social phenomena, London: UCL Press.

[6] Macaš, M., Lhotská, L. and Křemen, V., 2007, „Social Impact based Approach to Feature Subset Selection”, International Workshop on Nature Inspired Cooperative Strategies for Optimization, Studies in Computational Intelligence, Springer, in press.

[7] Nowak, A., J. Szamrej, and B. Latané, 1990, “From Private Attitude to Public-Opinion - a Dynamic Theory of Social Impact”, Psychological Review, 97(3), pp. 362-376.

[8] Williams, N. L. and Mitchell, M. (2005), “Investigating the success of spatial coevolutionary learning”, In Proceedings of the 2005 Genetic and Evolutionary Computation Conference, GECCO-2005, New York, ACM Press, pp. 523-530.

[9] Lewenstein, M., Nowak, A, and Latané, B. (1992), „Statistical Mechanics of Social Impact”, Physical Review A 45(2), pp.763-775.

[10] Bordogna, C.M. and Albano, E.V.(2007), ”Statistical Methods Applied to the Study of Opinion Formation Models: A Brief Overview and Results of a Numerical Study of a Model based on the Social Impact Theory”, Journal of Physics: Condensed Matter, 19(6).