A Fuzzy Cognitive Maps Modeling, Learning and Simulation Framework for Studying Complex System

A Fuzzy Cognitive Maps Modeling, Learningand Simulation Framework for Studying

Complex System

Maikel León1,∗, Gonzalo Nápoles1, Ciro Rodriguez2, María M. García1,Rafael Bello1, and Koen Vanhoof 3

1 Central University of Las Villas, Santa Clara, [email protected]

2 Cienfuegos University, Cienfuegos, Cuba3 Hasselt University, IMOB, Diepenbeek, Belgium

Abstract. This paper presents Fuzzy Cognitive Maps as an approachin modeling the behavior and operation of complex systems; they com-bine aspects of fuzzy logic, neural networks, semantic networks, expertsystems, and nonlinear dynamical systems. They are fuzzy weighted di-rected graphs with feedback that create models that emulate the behav-ior of complex decision processes using fuzzy causal relations. First, thedescription and the methodology that this theory suggests is examined,later some ideas for using this approach in the control process area arediscussed. An inspired on particle swarm optimization learning methodfor this technique is proposed, and then, the implementation of a toolbased on Fuzzy Cognitive Maps is described. The application of thistheory might contribute to the progress of more intelligent and indepen-dent systems. Fuzzy Cognitive Maps have been fruitfully used in decisionmaking and simulation of complex situation and analysis. At the end, acase study about Travel Behavior is analyzed and results are assessed.

Keywords: Fuzzy Cognitive Maps, Particle Swarm Optimization,Travel Behavior, Complex Systems, Decision Making, Simulation.

1 Introduction

Modeling dynamic systems can be hard in a computational sense and manyquantitative techniques exist. Well-understood systems may be open to any ofthe mathematical programming techniques of operations study. First, developingthe model usually requires a big deal of effort and specialized knowledge outsidethe area of interest. Secondly, systems involving important feedback may benonlinear, in which case a quantitative model may not be possible [1]. In the pastyears, conventional methods were used to model and control systems but theircontribution is limited in the representation, analysis and solution of complexsystems.

There is a great demand for the development of autonomous complex systemsthat can be achieved taking advantage of human like reasoning and description

J.M. Ferrández et al. (Eds.): IWINAC 2011, Part II, LNCS 6687, pp. 243–256, 2011.© Springer-Verlag Berlin Heidelberg 2011

244 M. León et al.

of systems [2]. Human way of thinking process for any method includes vaguedescriptions and can have slight variations in relation to time and space; for suchsituations Fuzzy Cognitive Maps (FCM) seem to be appropriate to deal with.FCM are a combination of some aspects from Fuzzy Logic, Neural Networksand other techniques; combining the heuristic and common sense rules of FuzzyLogic with the learning heuristics of the Neural Networks. They were introducedby Kosko [3], who enhanced cognitive maps with fuzzy reasoning, that had beenpreviously used in the field of socio-economic and political sciences to analyzesocial decision-making problems.

The use of FCM for many applications in different scientific fields was pro-posed, they had been apply to analyze extended graph theoretic behavior, tomake decision analysis and cooperate distributed agents, also were used as struc-tures for automating human problem solving skills and as behavioral models ofvirtual worlds, and in many other fields. That’s why, with the elaboration of aframework that allows the design, execution, analyses and simulation of FCMis provided to specialists of diverse knowledge areas of a means for the study,prediction and assessment of situations that characterize diverse real problems.

This work proposes such a computational tool, where is possible to representknowledge in a graphic and comprehensible way through FCM, to try to studythe systems like a whole, settling down how the entities that conform them areaffected with others, offering to users, not necessarily specialist in ComputerScience, a tool that allows the creation and execution of FCM, including exper-imentation facilities and as a framework for the study of complex real systems.

The paper is structured as follows. First, there is a section with a backgroundabout FCM; after that, it is discussed a procedure where a particle swarmoptimization metaheuristic is applied in the learning process of FCM, and apseudocode of the idea is also presented. Following this, a section with the com-putational tool is described, showing the main windows to give readers a bettercomprehension on what is offered. At the end a case study about Travel Behavioris analyzed, with important results of the application of FCM in the problemmodeling and an improvement over cognitive structure due to the use of thelearning method.

2 Overview about Fuzzy Cognitive Maps

FCM in a graphical illustration seem to be a signed directed graph with feedback,consisting of nodes and weighted arcs (see figure 1). Nodes of the graph placefor the concepts that are used to express the behavior of the system and theyare connected by signed and weighted arcs representing the causal relationshipsthat exist connecting the concepts [4]. It must be mentioned that the values inthe graph are fuzzy, so concepts take values in the range between [0, 1] and theweights of the arcs are in the interval [-1, 1]. The weights of the arcs betweenconcept Ci and concept Cj could be positive (Wij > 0) which means that anaugment in the value of concept Ci leads to the increase of the value of conceptCj, and a decrease in the value of concept Ci leads to a reduce of the value

A Fuzzy Cognitive Maps Modeling, Learning and Simulation Framework 245

Fig. 1. Simple Fuzzy Cognitive Map. Concept activation level.

of concept Cj. Or there is negative causality (Wij < 0) which means that anincrease in the value of concept Ci leads to the decrease of the value of conceptCj and vice versa.

Observing this graphical representation, it becomes clear which concept in-fluences other concepts showing the interconnections between concepts and itpermits updating in the construction of the graph. Each concept represents acharacteristic of the system; in general it stands for events, actions, goals, values,trends of the system that is modeled, etc. Each concept is characterized by anumber that represents its value and it results from the renovation of the realvalue of the system’s variable [5].

Beyond the graphical representation of the FCM there is its mathematicalmodel. It consists of a 1×n state vector A which includes the values of the nconcepts and a n×n weight matrix W which gathers the weights Wij of theinterconnections between the n concepts. The value of each concept is influencedby the values of the connected concepts with the appropriate weights and by itsprevious value. So the value Ai for each concept Ci can be calculated, amongother possibilities, by the following rule expressed in (1).

Ai = f

⎛⎜⎝

n∑j=1

j is not i

AjWji

⎞⎟⎠ (1)

Where Ai is the activation level of concept Ci, Aj is the activation level of conceptCj and Wji is the weight of the interconnection between Cj and Ci, it is to say,the value of Ai depends of the weighted sum of its input concepts, and f is athreshold or normalization function. So the new state vector Anew is computedby multiplying the previous state vector Aold by the weight matrix W, see (2).The new vector shows the effect of the change in the value of one concept in thewhole FCM [6].

Anew = f (AoldW ) (2)

246 M. León et al.

In order to build an FCM, the knowledge and experience of one expert onthe system’s operation must be used. The expert determines the concepts thatbest illustrate the system; a concept can be a feature of the system, a state ora variable or an input or an output of the system; identifying which factors arecentral for the modeling of the system and representing a concept for each one.

Moreover the expert has observed which elements of the system influenceothers elements; and for the corresponding concepts the expert determines thenegative or positive effect of one concept on the others, with a fuzzy value foreach interconnection, since it has been considered that there is a fuzzy degree ofcausation between concepts.

It is possible to have better results in the drawing of the FCM, if more thanone expert is used. In that case, all experts are polled together and they de-termine the relevant factors and thus the concepts that should be presented inthe map. Then, experts are individually asked to express the relationship amongconcepts; during the assigning of weights, three parameters must be considered:how strongly concepts influence each other, what is the sign of the weight andwhether concepts cause [7].

This is one advantage over other approaches like Petri Nets (PN) or BayesianNetworks (BN). PN are a graphical and mathematical modeling tool consist-ing of places, transitions, and arcs that connect them, that can be used as avisual-communication aid similar to flow charts, block diagrams, and networks.As a mathematical instrument, it is possible to set up state equations, alge-braic equations, and other mathematical models governing the performance ofsystems.

It is well known that the use of PN has as a disadvantage the drawing processby a non-expert in this technique, that’s way there is a limited numbers of toolsusable for this purpose, and it is not well established how to combine differentPN that describe the same system [8].

FCM feedback structure also makes a distinguishing from the earlier forward-only acyclic cognitive maps and from modern Artificial Intelligence expert-systemsearch trees. Such tree structures are not dynamical systems because they lackedge cycles or closed inference loops. Nor are trees closed under combination.Combining several trees does not produce a new tree in general because cyclesor loops tend to occur as the number of combined trees increases.

But combining FCM always produces a new FCM. The combined FCM nat-urally averages the FCM and their corresponding causal descriptions as well asmuch of their dynamics (example on figure 2).The user can combine any numberof weighted FCM, combining into a single averaged FCM by the simple artificeof adding their scaled and augmented (zero-padded) adjacency edge matrices.

The strong law of large numbers then ensures that the sample average ofeven quantized or rounded-off FCM will converge with probability one to theunderlying but often unknown FCM that generates these matrix realizations.So FCM edge-matrix combination improves with sample size. FCM knowledgerepresentation likewise tends to improve as the user combines more FCM froman ever larger pool of domain experts.


Fig. 2. Combining two FCM

If there will be a collection of individual FCM that must be combined intoa collective map (see figure 3) and if there are experts of different credibil-ity, for them, then their proposed maps must be multiplied with a nonnegative“credibility” weight. So the combination of these different FCM will produce anaugmented FCM.

Fig. 3. Combining some FCM into a collective map

As over PN, this is an advantage over BN [9]. BN is a powerful tool for graphi-cally representing the relationships among a set of variables and for dealing withuncertainties in expert systems, but demanding effort caused by specification ofthe net (structure and parameters) and an expensive algorithm of propagationof probabilities [10]. Also is not evident for a non-expert in this field how toconstruct a BN, and even more difficult how to compare or combine differentBN that describe the same system.

When a FCM has been constructed, it can be used to model and simulate thebehavior of the system. Firstly, the FCM should be initialized, the activation level

248 M. León et al.

of each of the nodes of the map takes a value based on expert’s opinion for thecurrent state and then the concepts are free to interact. This interaction betweenconcepts continues until a fixed equilibrium is reached; a limited cycle is reachedor a chaotic behavior is exhibited. So, FCM are a powerful methodology thatcan be used for modeling systems, avoiding many of the knowledge extractionproblems which are usually present in by rule based systems [11].

The threshold or normalization function used over concept value of FCM,serves to decrease unbounded inputs to a severe range. This destroys the pos-sibility of quantitative results, but it gives us a basis for comparing nodes (onor off, active or inactive, etc.). This mapping is a variation of the “fuzzification”process in fuzzy logic, giving us a qualitative model and frees us from strictquantification of edge weights [12].

3 A FCM Learning Process Using PSO Metaheuristic

Problems associated with manual development of FCM encourage researchersto work on automated or semi-automated computational methods for learningFCM structure using historical data. Semiautomated methods still require arelatively limited human intervention, whereas fully automated approaches areable to compute a FCM model solely based on historical data.

Researches on learning FCM models from data have resulted in a number of al-ternative approaches. One group of methods is aimed at providing a supplementtool that would help experts to develop accurate model based on their knowledgeabout a modeled system. Algorithms from the other group are oriented towardeliminating human from the entire development process, only historical data arenecessary to establish FCM model [13].

Particle Swarm Optimization (PSO) method, which belongs to the class ofSwarm Intelligence algorithms, can be used to learn FCM structure based onhistorical data, consisting of a sequence of state vectors that leads to a desiredfixed-point attractor state.

PSO is a population based algorithm, which goal is to perform a search bymaintaining and transforming a population of individuals. This method improvesthe quality of resulting FCM model by minimizing an objective or heuristicfunction. The function incorporates human knowledge by adequate constraints,which guarantee that relationships within the model will retain the physicalmeaning defined by experts.

The following idea flow chart (see figure 4) illustrates the application of PSOin the readjusting of the weight matrix, trying to find a better configurationthat guaranty a convergence or waited results. PSO is applied straight forwardlyusing an objective function defined by the user. Each particle of the swarm is aweight matrix, encoded as a vector.

First the concepts and relation are defined, and the construction of FCM ismade, and then is possible to make simulations and obtain outputs due to theinference process. If the new values are not adequate, known by the execution ofthe heuristic function, then it is necessary a learning process (in this case throughthe use of PSO metaheuristic) having as results new values for the weight matrix.


Fig. 4. Application of PSO metaheuristic as a FCM learning method

Fig. 5. Pseudocode of the PSO proposed method

Mixed approaches, like using Genetic Algorithm and Particle Swarm Opti-mization have been performed so far. Literature reports some results that arereally very promising and encouraging further researches and applications in thisarea.

Using this approach, new zones of the search space are explored in a particularway, through the crossover of good initial particles and the mutation of someothers, just to mention two posible approaches. In the following pseudocode

250 M. León et al.

illustrated in figure 5, we can appreciate the general philosophy of our proposedmethod. In this case the genetic algorithm operators are used as initial steps.

4 Tool Based on Fuzzy Cognitive Maps

The scientific literature shows some software products developed with the inten-tion of drawing FCM by non-expert in computer science, as FCM Modeler [14]and FCM Designer [15]. The first one is a rustic incursion, while the second oneis a better implementation, but still hard to interact with and it doesn’t haveexperimental facilities. Figure 6 shows the general architecture of our propos-ing tool to model and simulate FCM, the organization and structuring of thecomponents are presented.

Fig. 6. General architecture of the tool

In figure 7 it is possible to observe the main window of the tool, and a mod-eled example, in the interface appears some facilities to create concepts, makerelations, and define parameters, also to initialize the execution of the inferenceprocess, and visualization options for a better understanding of the simulationprocess. There were defined some facilities and options in the tool, to create,open or save an FCM. Through these amenities a non-expert in Computer Sci-ence is able to elaborate FCM describing systems; we had paid attention to thesefacilities guarantying an usable tool, specifically for simulation purposes.

In figure 8 we can appreciate some important options, where is possible todefine the assignment of a delay time in the execution for a better understandingof the running of the FCM in the inference process, also it is possible to define thenormalization function that the FCM will use in the running [16]. In simulationexperiments the user can compare results using these different functions or justcan select the appropriate function depending of the problem to model:


Fig. 7. Main view of FCM Tool

Fig. 8. Run options of the FCM Tool

252 M. León et al.

- Binary FCM are suitable for highly qualitative problems where only repre-sentation of increase or stability of a concept is required.

- Trivalent FCM are suitable for qualitative problems where representation ofincrease, decrease or stability of a concept is required.

- Sigmoid FCM are suitable for qualitative and quantitative problems whererepresentation of a degree of increase, a degree of decrease or stability of aconcept is required and strategic planning scenarios are going to be introduced.

On the other hand, there are also options for experts on the topic, specificallyrelated to the learning process, as it is possible to perceive in figure 9. Thenecessary definition of parameters is done through this window.

In simulation and experiment in general, the visualization consists a funda-mental aspect, that´s why it was consevied a panel where the learning processcan be observed, figure 10 shows the heuristic function evaluation panel.

Fig. 9. Parameters definition for PSO method

Fig. 10. Heuristic function evaluation in the learning process


It is possible to see how the FCM is updated with a new weight matrix thatbetter satisfy the waited results, in less than 100 iteration, there is a significativeconvergence.

5 Case Study: Modeling Travel Behavior

In the process of transportation planning, travel demand forecast is one of themost important analysis instruments to evaluate various policy measures aimingat influencing travel supply and demand. In past decades, increasing environ-mental awareness and the generally accepted policy paradigm of sustainabledevelopment made transportation policy measures shift from facilitation to re-duction and control.

Objectives of travel demand management measures are to alter travel behaviorwithout necessarily embarking on large-scale infrastructure expansion projects,to encourage better use of available transport resources avoiding the negativeconsequences of continued unrestrained growth in private mobility [17].

Individual activity travel choices can be considered as actual decision prob-lems, causing the generation of a mental representation or cognitive map of thedecision situation and alternative courses of action in the expert’s mind. Thiscognitive map concept is often referred to in theoretical frameworks of traveldemand models, especially related to the representation of spatial dimensions,but much features can be taken into account.

However, actual model applications are scarce, mainly due to problems inmeasuring the construct and putting it into the model’s operation. The devel-opment of the mental map concept can benefit from the knowledge provided byindividual tracking technologies [18]. Researches are focusing on that direction,in order to improve developed models and to produce a better quality of systems.

At an individual level it is important to realize that the relationship betweentravel decisions and the spatial characteristics of the environment is establishedthrough the individual’s perception and cognition of space. As an individualobserves space, for instance through travel, the information is added to theindividual’s mental maps (see figure 11).

In the city of Hasselt, Belgium, a study related to travel behavior was made.More than 200 real habitants were asked to specify how they take into accountthe transport mode they will use for an imaginary shopping activity: situationvariables, attributes, benefits; and starting from that data, a FCM structure perperson was constructed.

At the same time, virtual scenarios were presented, and the personal decisionswere stored. Figure 12 shows the acting of the initial modeled FCM, for example,only the 24% of the maps were able to predict 100% scenarios. A FCM learningmethod, based on the PSO metaheuristic was applied, having the stored scenar-ios as training data, and the results show that after the learning process, 77% ofmaps were able to predict 100% of scenarios.

It is considered a significant improvement over the maps, having structuresable to simulate how people think when visiting the city center, specifically the

254 M. León et al.

Fig. 11. Abstraction levels of mind related to Travel Behavior

Fig. 12. Improvement made to data

transport mode they will use (car, bus or bike), offering policy makers, a toolto play with, in order to test new policies, and to know in advance the possibleresounding in the society (buses cost, parking cost, bike incentive, etc.).

6 Conclusions

It has been examined Fuzzy Cognitive Maps as a theory used to model thebehavior of complex systems, where is extremely difficult to describe the entiresystem by a precise mathematical model. Consequently, it is more attractiveand practical to represent it in a graphical way showing the causal relationshipsbetween concepts. Since this symbolic method of modeling and control of asystem is easily adaptable and relies on human expert experience and knowledge,it can be considered intelligent.

A learning algorithm for determining a better weight matrix for the through-put of FCM was presented. It is an unsupervised weight adaptation methodologythat have been introduced to fine-tune FCM causal links and accompanied withthe good knowledge of a given system or process can contribute towards the


establishment of FCM as a robust technique. Experimental results based on sim-ulations of the process system verify the effectiveness, validity and advantageousbehavior of the proposed algorithm.

The area of FCM learning is very promising because the FCM obtained aredirectly interpretable by a human and are an useful tool to extract informationfrom data about the relations between the concepts or variables inside a domain.

The development of a tool based on FCM for the modeling of complex systemswas presented, showing the facilities for the creation of FCM, the definition ofparameters and options to make the inference process more comprehensible,understanding and used for simulations experiments.

At the end, a real case study was presented, showing a possible Travel Be-havior modeling through FCM, and the benefits of the application of a learningmethod inspired in the PSO metaheuristic, obtaining an improvement on theknowledge structures originally modeled. In this shown example a social andpolitic repercussion is evident, as we offer to policymakers a framework and realdata to play with, in order to study and simulate individual behavior and pro-duce important knowledge to use in the development of city infrastructure anddemographic planning.

References

1. Kosko, B.: Neural Networks and Fuzzy systems, a dynamic system approach tomachine intelligence, p. 244. Prentice-Hall, Englewood Cliffs (1992)

2. Parpola, P.: Inference in the SOOKAT object-oriented knowledge acquisition tool.Knowledge and Information Systems (2005)

3. Kosko, B.: Fuzzy Cognitive Maps. International Journal of Man-Machine Stud-ies 24, 65–75 (1986)

4. Koulouritios, D.: Efficiently Modeling and Controlling Complex Dynamic Systemsusing Evolutionary Fuzzy Cognitive Maps. International Journal of ComputationalCognition 1, 41–65 (2003)

5. Wei, Z.: Using fuzzy cognitive time maps for modeling and evaluating trust dy-namics in the virtual enterprises. Expert Systems with Applications, 1583–1592(2008)

6. Xirogiannis, G.: Fuzzy Cognitive Maps as a Back End to Knowledge-based Sys-tems in Geographically Dispersed Financial Organizations. Knowledge and ProcessManagement 11, 137–154 (2004)

7. Aguilar, J.: A Dynamic Fuzzy-Cognitive-Map Approach Based on Random NeuralNetworks. Journal of Computational Cognition 1, 91–107 (2003)

8. Li, X.: Dynamic Knowledge Inference and Learning under Adaptive Fuzzy PetriNet Framework. IEEE Transactions on Systems, Man, and Cybernetics Part C:Applications and reviews (2000)

9. Castillo, E.: Expert Systems and Probabilistic Network Models. Springer,Heidelberg (2003)

10. Intan, R.: Fuzzy conditional probability relations and their applications in fuzzyinformation systems. Knowledge and Information Systems (2004)

11. Carlsson, C.: Adaptive Fuzzy Cognitive Maps for Hyperknowledge Representationin Strategy Formation Process. In: IAMSR, Abo Akademi University (2005)

256 M. León et al.

12. Stylios, C.: Modeling Complex Systems Using Fuzzy Cognitive Maps. IEEE Trans-actions on Systems, Man and Cybernetics 34, 155–162 (2004)

13. Mcmichael, J.: Optimizing Fuzzy Cognitive Maps with a Genetic Algorithm AIAA1stIntelligent Systems Technical Conference. Chicago, Illinois (2004)

14. Mohr, S.: Software Design for a Fuzzy Cognitive Map Modeling Tool. TensselaerPolytechnic Institute (1997)

15. Contreras, J.: Aplicación de Mapas Cognitivos Difusos Dinámicos a tareas de su-pervisión y control. Trabajo Final de Grado. Universidad de los Andes. Mérida,Venezuela (2005)

16. Tsadiras, A.: A New Balance Degree for Fuzzy Cognitive Maps. Technical Report.Department of Applied Informatics. University of Macedonia (2007)

17. Gutiérrez, J.: Análisis de los efectos de las infraestructuras de transporte sobrela accesibilidad y la cohesión regional. Estudios de Construcción y Transportes.Ministerio de Fomento. España (2006)

18. Wu, Q.: Multiknowledge for decision making. Knowledge and Information Systems(2005)

A Fuzzy Cognitive Maps Modeling, Learning and Simulation Framework for Studying Complex System

Documents

Transcript of A Fuzzy Cognitive Maps Modeling, Learning and Simulation Framework for Studying Complex System