Applied Energy 130 (2014) 384–395

Contents lists available at ScienceDirect

Applied Energy

journal homepage: www.elsevier .com/ locate/apenergy

Grid-price-dependent energy management in microgrids usinga modified simulated annealing triple-optimizer

http://dx.doi.org/10.1016/j.apenergy.2014.05.0600306-2619/� 2014 Elsevier Ltd. All rights reserved.

⇑ Corresponding author. Tel.: +43 424256300257.E-mail address: first.last@ctr.at (R. Velik).

Rosemarie Velik ⇑, Pascal NicolayCTR Carinthian Tech Research, Europastrasse 4/1, 9524 Villach, Austria

h i g h l i g h t s

� Proposes a modified simulated annealing triple-optimizer for energy management.� Optimizes gain of microgrids participating in a market with variable grid prices.� Provides optimal or close to optimal solutions with limited computation effort.

a r t i c l e i n f o

Article history:Received 10 March 2014Received in revised form 26 May 2014Accepted 27 May 2014

Keywords:Renewable energyBattery storageEnergy managementVariable grid pricesMicrogridsOptimization

a b s t r a c t

This article introduces a modified simulated annealing triple-optimizer for finding the optimal energymanagement strategy in terms of financial gain maximization in grid-connected, storage-augmented,photovoltaics-supplied prosumer building microgrids in a variable grid price scenario. For evaluatingthe performance of the optimizer, a number of test cases are specified offering different trading optionsto the prosumers. Obtained results are compared to a total state space search reference method and dem-onstrate that the simulated annealing approach was for all test cases able to find a globally optimal orclose to optimal solution in significantly less computation time than the total space search referencemethod.

� 2014 Elsevier Ltd. All rights reserved.

1. Introduction

The structural change from centralized (electric) energy pro-duction to distributed renewable energy production systems leadsto a range of new challenges [1–3]. In this context, many stake-holders emphasize on the importance of the transformation ofour current power grid to a smart grid via the massive integrationof measurement, information, communication, and control tech-nology [4]. Furthermore, to increase manageability and reducetransportation losses, the division of the smart grid into sub-sys-tems, so-called smart microgrids, is suggested [5,6]. Microgridsare low voltage distribution networks comprising energy genera-tors (e.g. photovoltaics systems), storage elements (e.g. batteries),and loads [7,8]. A microgrid can either be run in island mode orit can be connected to other microgirds and/or the main grid andexchange energy with them.

In grid-connected microgrids, electric power exchange with themain grid has traditionally been based on a constant, pre-specifiedprice per kW h. However, with the integration of smart metre tech-nologies, capable of accurately measuring energy consumption andproduction in each time instant, a shift to demand- and supply-sensitive pricing models is likely to occur in the near future[9,10]. Following this development, microgrid operators willbecome motivated to adapt their energy trading behaviour withthe main grid and/or other microgrids according to the currenttrading conditions in order to maximize their financial benefits.To do so without having to take the significant burden of con-stantly having to select the best actions to take in the given situa-tion, specific energy management systems will have to be put intoplace [11–13]. In this context, finding optimal energy managementstrategies for particular system configurations and their operators’objectives will become an important topic, which can massivelybenefit from computational optimization methods [14–16].

In this article, we propose a modified simulated annealing tri-ple-optimizer approach for automatically finding the best financialgain optimization strategy in a microgrid embedded into a variable

R. Velik, P. Nicolay / Applied Energy 130 (2014) 384–395 385

grid price environment while at the same time assuring load sup-ply and the full exploitation of available photovoltaics energy. Themircorgrid emulates a typical prosumer household containing ele-ments like loads, a photovoltaics system and battery storage. It isconnected to the main grid as well as to other mircrogrids repre-senting neighbouring prosumer buildings. To evaluate the perfor-mance of this method, it is compared to a total state spacesearch approach for various specified test cases.

2. Related work

The topic of cost reduction/optimization in microgrids viaenergy management has already been addressed by severalresearch groups in different contexts.

Concerning the topic of microgrid operation in island mode, Pari-sio et al. [17] develop a model predictive control approach for costefficient microgrid operation management and perform a feasibil-ity study by applying this method to an experimental microgridlocated in Athens. Marzband et al. [18] propose a control architec-ture for cost efficient power generation and uninterruptable powersupply services in an islanded microgrid. In Marzband et al. [19],the same research group discusses a gravitational search optimiza-tion method for performance improvement (e.g., peak consump-tion reduction, electricity generation cost minimization) inisolated microgrids. Chen et al. [20] present an energy manage-ment model, including sensitivity analyses for storage capacityinvestment and electricity demand growth, for determining opti-mal operating strategies for profit maximization in a microgridsystem in Taiwan. Matamoros et al. [6,21] address the minimiza-tion of total costs for energy generation and transportation oftwo microgrids that can exchange energy with each other but areisolated from the main power grid. For this purpose, a centraland a distributed control approach are investigated using an ana-lytical convex optimization technique and an iterative approach.

Targeting the topic of energy trading of a microgrid with the maingrid, Celli et al. [22] focus on the development of a neural networkbased energy management system to determine the correct dis-patch of generators in a microgrid to participate in the energy mar-ket and minimize global energy costs. Livengood et al. [10]introduce a control device called ‘‘Energy Box’’ for managing homeor small business electrical energy use in an environment ofdemand-sensitive real-time pricing. Based on forecast informationfrom weather, grid price, and occupancy level, it aims at managingthe usage, storage and selling of energy by employing a stochasticdynamic programming method. Mashhour and Moghaddas-Tafre-shi [23] propose a multi-period optimization model based on hier-archical control for a microgrid setup capable of participating tothe wholesale energy market as both producer and consumer withthe objective of maximizing revenues and minimizing costs. Kriettand Salani [24] target a residential microgrid (including electricaland thermal energy, energy storages, and partly controllable loads)and proposes a generic mixed integer linear programmingapproach for its operating cost minimization in marked-basedprice scenarios.

Furthermore, concepts and models have been introduced aim-ing at multi-objective optimization in microgrids. In this context,Mohamed and Koivo [25] aim at presenting a generalized formula-tion for maintenance cost optimization and emission reduction of amircrogrid (consisting of a wind turbine, a fuel cell, a diesel gener-ator, a photovolatics array, a battery storage, and loads) based ongame theory. Similarly, Chaouachi et al. [26] introduce a microgridenergy management approach for operation cost and emissionminimization using a linear-programming based multi-objectiveoptimization method taking into account the predicted availabilityof renewable energy resources and load demand. Zhao et al. [27]

present a genetic algorithm for optimal unit sizing of a stand-alonemicrogrid considering multiple objectives including life-cycle costminimization, renewable energy source penetration maximization,and emission minimization. Costa and Kariniotaki [7] propose astochastic optimization method for microgrid operation to opti-mize local energy production, storage, and the exchange with themain grid according to market conditions. In Niknam et al. [28],again a stochastic model for optimal energy management in agrid-connected microgrid is presented focusing on cost and emis-sion minimization. Constantopoulos et al. [29] introduce a conceptfor a certainty equivalent control scheme that aims at exploitinginformation on room temperature, grid price, and weather toobtain a balance between energy costs and room comfort underspot electricity pricing conditions. Houwing et al. [30] address asetup in which a household can generate its own power with amicro-combined heat and power plant, store produced heat andelectricity, and trade them with an external energy supplier. Thesystem aims at an operational cost minimization while at the sametime not violating residential electricity and heat requirementsusing a model predictive control strategy.

The main novelty of the article at hand is that it investigates amulti-objective optimization method for finding cost-optimalmicrogrid operation strategies not only featuring energy tradingwith the main grid under time-variable grid price conditions butadditionally allowing energy exchange with five neighbouring mir-crogrids according to agreed energy exchange conditions. One par-ticularity of the optimizer is that it does not have to find just oneoptimal operation mode but has to provide different energy man-agement strategies for the different occurring grid prices. Due tothe use of storage elements, the strategies for these different gridprices are actually interdepent. Finding an optimal solution in sucha setup requires the search through a relatively large search space.Accordingly, an optimization method has to be developed that ison the one hand capable of finding an optimal solution in a largesearch space without suffering from the common problem of get-ting easily ‘‘trapped’’ in non-optimal local minima but on the otherhand computationally not too expensive to suffer from the ‘‘curseof dimensionality’’ [31].

3. Materials and methods

3.1. System configuration and optimization challenge definition

Fig. 1 illustrates the microgrid system configuration for which agrid-price dependent set of optimal energy management strategiesshall be found. The mircogrid represents a prosumer householdbuilding consisting of (1) household loads, (2) a photovoltaic (PV)system, (3) a battery storage system, (4) a connection to the maingrid, and (5) an interface to five neighbouring microgrids repre-senting further prosumer buildings. The same system configura-tion as depicted in Fig. 1 also applies to each of the fiveneighbouring buildings.

Fig. 1 additionally indicates the principally possible directionsof energy exchange between individual components. These energyexchange options correspond to the actions that can be selected inorder to optimize the microgrid’s energy management strategy.Table 1 summarizes these actions together with the pre-conditionsthat have to be fulfilled for their execution. The purpose of theenergy optimizer presented in this paper is the grid-price depen-dent ranking of these actions according to their execution priorityfor achieving a financial gain maximization within the microgridvia energy trading (see Section 3.5.1) in different specified test sce-narios (see Section 3.4).

The action with the ID 1 (‘‘Do No More Activity’’) is selected if,after having selected a certain set of actions, no further actions

Fig. 1. System components of test setup including possible directions of energy exchange (depicted as arrows).

386 R. Velik, P. Nicolay / Applied Energy 130 (2014) 384–395

shall be carried out. In this case, all actions selected and rankedafter action 1 are ignored and remain without execution. For therest of the actions specified in Table 1, it was specified that in caseof their execution, always the maximum possible amount ofenergy is exchanged between their source and sink before callingthe next action. For example, for the action ‘‘Own PV Energy toNeighbour Loads’’, this could result in the following two cases: ifless PV energy is remaining in the household than is needed bythe loads of a specific neighbour household, all remaining availablePV energy of this household would be directed to the neighbourloads; on the other side, if more PV energy is available than neededby these neighbour loads, only the amount of PV energy would bedirected to the neighbour household that is necessary to supplythese loads.

3.2. Test data acquisition and pre-processing

The development and evaluation of our energy optimizerrequired the acquisition of realistic PV, load, and grid price dataover a period of 30 days. For this purpose, a simple renewableenergy data generator was used, which was developed during thenationally funded, multi-firm research project Vision Step I – SmartCity Villach [32,33]. For data acquisition, a resolution of always onehour was chosen.

The Figs. 2–4 give examples of generated load, PV, and gridprice curves for the six household buildings over a period of always24 h. While for each household an individual load curve was gen-erated, the same PV profile was used for the six household build-ings. This corresponds to the case that all buildings have thesame roof orientation and tilt angle and the same number and typeof installed PV modules. Such a constellation can for instance be

found in residential estates realized by larger construction compa-nies. Such companies could also be the right stakeholders forsetting up the infrastructure for an energy exchange betweenneighbouring microgrids. The load and PV curves were normalizedto yield and average load consumption and PV production of16 kW h per household and day when averaging over the 30 daysrecording period. Accordingly, over the 30 days, for each building,the amount of consumed energy equaled the amount of producedPV energy. The generated grid price curve was employed for energyexchange between each household and the grid for both purchas-ing and selling energy and varied between the discrete values 1,2 and 3 units/kW h with a 24-h average of always 2 units/kW h.

3.3. System modeling

The simulation of the system configuration introduced in Fig. 1was realized using Visual Studio C++. To simplify the system anal-ysis and keep modeling simple, efficiency losses of system compo-nents were neglected for the present study. This allowed atechnology-independent analysis of the theoretically possibleupper limits of financial gain maximization of particular energyoptimization strategies.

For the test scenarios including battery storage units, simula-tions were carried out with a usable (not nominal) storage capacityof 16 kW h per household, which equaled the average amount ofdaily load consumption and PV production per building. At systemstartup, all battery storages were empty.

Concerning energy exchange with the main grid, energy wasalways traded (purchased or sold) according to the current gridprice (see Section 3.2). For energy exchange with neighbours,time-independent trading tariffs were fixed equaling 1 unit/kW h

Table 1Actions according to supported directions of energy exchange between system components.

Action ID Action description Pre-conditions for execution

1 Do no more activity None2 Own PV energy to own loads Own PV energy remaining

Own loads still need energy3 Own PV energy to own storage Own PV energy remaining

Own storage is not full4 Own stored energy to own loads Own storage is not empty

Own loads still need energy5 Own PV energy to grid Own PV energy remaining6 Grid energy to own loads Own loads still need energy7 Own PV energy to neighbour loads Own PV energy remaining

Neighbour loads still need energyMutual agreement for according energy exchange

8 Neighbour PV energy to own loads Neighbour PV energy remainingOwn loads still need energyMutual agreement for according energy exchange

9 Own PV energy to neighbour storage Own PV energy remainingNeighbour storage is not fullMutual agreement for according energy exchange

10 Neighbour PV energy to own storage Neighbour PV energy remainingOwn storage is not fullMutual agreement for according energy exchange

11 Own stored energy to neighbour loads Own storage is not emptyNeighbour loads still need energyMutual agreement for according energy exchange

12 Neighbour stored energy to own loads Neighbour storage is not emptyOwn loads still need energyMutual agreement for according energy exchange

13 Own stored energy to grid Own storage is not empty14 Grid energy to own storage Own storage is not full

Fig. 2. Representative load profile example of the six test households over a periodof 24 h.

Fig. 3. Representative PV profile example over a period of 24 h.

Fig. 4. Representative grid price profile example over a period of 24 h.

Fig. 5. Specified order of neighbour energy trading between buildings.

R. Velik, P. Nicolay / Applied Energy 130 (2014) 384–395 387

for energy from the PV system and 2 units/kW h for energy fromthe battery storage. To exchange energy between two neighbours,both neighbours had to agree on the specific energy exchange bysending offers and requests. If more than one neighbour was

Table 2Specified test scenarios and corresponding selectable actions.

Sce-narioID

Selectable actions

1: Donomoreactivity

2: OwnPVenergyto ownloads

3: OwnPVenergyto ownstorage

4: Ownstoredenergyto ownloads

5: OwnPVenergyto grid

6: Gridenergyto ownloads

7: Own PVenergy toneighbourloads

8:NeighbourPV energyto ownloads

9: Own PVenergy toneighbourstorage

10:NeighbourPV energyto ownstorage

11: Ownstoredenergy toneighbourloads

12:Neighbourstoredenergy toown loads

13:Ownstoredenergyto grid

14:Gridenergyto ownstorage

Gain maximization via energy exchange with main grid1 Individual PV

use and nostorage use

x x x x

2a Individual PVand storage use

x x x x x x

2b Individual PVand storage use

x x x x x x x x

Gain maximization via energy exchange with main grid and neighbours (unified neighbour group)3 Individual PV

use, neighbourPV exchange forloads, and nostorage use

x x x x x x

4a Individual PVand storage useand neighbourPV exchange forloads

x x x x x x x x

4b Individual PVand storage useand neighbourPV exchange forloads

x x x x x x x x x x

5a Individual andneighbour PVand storage use

x x x x x x x x x x x x

5b Individual andneighbour PVand storage use

x x x x x x x x x x x x x x

Gain maximization via energy exchange with main grid and neighbours (mixed neighbour group)6 Individual PV

use, neighbourPV exchange forloads, and nostorage use

x x x x x x

7a Individual PVand storage useand neighbourPV exchange forloads

x x x x x x x x

7b Individual PVand storage useand neighbourPV exchange forloads

x x x x x x x x x x

8a Individual andneighbour PVand storage use

x x x x x x x x x x x x

8b Individual andneighbour PVand storage use

x x x x x x x x x x x x x x

388 R. Velik, P. Nicolay / Applied Energy 130 (2014) 384–395

interested in selling or purchasing energy, trading priorities fol-lowed a fixed order as specified in Fig. 5. First, the energy in ques-tion was always offered to the direct next neighbour (round 1). Ifthis neighbour was not interested in the exchange, the neighbournext in line got this opportunity (round 2) and so on.

3.4. Investigated test scenarios

The objective of the developed optimizer (see Section 3.5) is themaximization of the financial gain of a household while at thesame time assuring that all its loads are supplied with sufficient

energy and no PV energy is wasted. To evaluate the performanceof the optimizer, a range of different test scenarios was specified,which are summarized in Table 2. In each scenario, different actionoptions could be selected.

The scenarios 1, 2a, and 2b are concerned with finding energyoptimization strategies via interaction with the main grid and withor without storage use. The scenarios 3–8b additionally supportdifferent energy exchange options between neighbours. Concern-ing the storage use, all scenarios labeled as Xa feature a chargingof the storage just from the own or the neighbour PV system.The discharging can be performed by directing the energy to the

Fig. 6. Algorithm blocks of optimizer.

Table 3Optimization sub-objectives and their assigned variables, target values, and priorities.

Sub-objective Optimization variable Target value Priority

All active loads need to be supplied with sufficient energy Remaining_Necessary_Load_Energy 0 1Financial gain obtained through energy trading with neighbours and the grid should be maximal Financial_Gain Max 2No PV energy should remain unused PV_Energy_Remaining 0 3

R. Velik, P. Nicolay / Applied Energy 130 (2014) 384–395 389

own or neighbour loads or selling it to the main grid. The scenarioslabeled as Xb additionally feature a charging and discharging of thestorage by energy exchange with the main grid.

In the scenarios 3–5b, all six households of the neighbourhoodaim at financial gain maximization and thus constitute a unifiedneighbourhood group. In contrast to this, the scenarios 6–8b emu-late the case of a mixed neighbourhood group in which half of thehouseholds aim at finding a financial gain maximization strategywhile the other half follows the objective of PV energy self-con-sumption maximization within the neighbourhood by applyingan energy management strategy as specified in detail in [12]. Inprinciple, this self-consumption maximization strategy aims toconsume as much of the PV energy produced within the neigh-bourhood as possible locally and exchange as little energy as pos-sible with the main grid.

3.5. Optimizer

To find a grid-price dependent set of action rankings leading tothe maximum financial gain via energy trading, the optimizer wassplit into two main blocks as depicted in Fig. 6, which cyclicallyexchange information until an optimal solution is found. The firstblock, described in detail in Section 3.5.1, concerns the calculationof the optimization function based on a current set of action orders,which is applied to the specified 30-day PV, load, and grid priceprofiles and additionally considers in each time instant informa-tion about the status of the battery storage and energy exchangeoffers and requests from neighbour buildings. The second block,described in detail in Section 3.5.2, concerns the search of the opti-mal action order set for a particular scenario based on the latestresult of the optimization function. As the grid price in the speci-fied scenarios varies between the discrete values 1, 2, and 3, a sep-arate action order ranking has to be determined for each grid priceleading to a triple-set of action orders. One particularity of the sys-tem, resulting from the use of storage elements, is that the actionrankings for the different grid prices are not independent. Whatis an optimal ranking for one particular grid price depends on what

the rankings look like for the two other gird prices. For finding theoptimal triple-set of action orders, two different optimizing algo-rithms were implemented, which are described in detail in the Sec-tions 3.5.2.1 and 3.5.2.2.

3.5.1. Optimization function calculationAs illustrated in Fig. 6, the calculation of the optimization func-

tion is divided into three parts (see also Table 4):

3.5.1.1. Part I: action execution & protocolling of energy supply/consumption. Based on the current action order set, the corre-sponding energy exchange is performed between the system com-ponents for each time instant i and for each building j. The amountof energy exchanged between the different sources and sinks,which depends on the current PV production and load consump-tion rates, the battery storage status, and energy exchange offersand requests from neighbours, is protocolled in correspondingvariables source_x_to_sink_y[i][j].

3.5.1.2. Part II: calculation of sub-objective variables. The commonobjectives of all scenarios listed in Table 2 are (1) assuring thatall loads in the household are supplied with sufficient energy whileat the same time (2) maximizing the financial gain of a householdvia grid-price dependent energy trading and (3) taking care that noproduced PV energy is wasted (by not exploiting it for anypurpose).

In Table 3, a description of these sub-objectives, their assignedvariable names, their ideal ‘‘target values’’, and their priority islisted. The first priority is given to the supply of all loads. Nexthighest importance is assigned to the financial gain maximization.The assurance of not wasting produced PV energy has third prior-ity. In part II of the optimization algorithm, the variables represent-ing these different sub-objectives are calculated as average valuesper building and day from the protocolled energy exchange ratesdetermined in part I. For the scenarios 1–5b, the average was takenover all 6 buildings as for all of them the same optimization strat-

Table 4Pseudo-code for the calculation of the optimization function.

390 R. Velik, P. Nicolay / Applied Energy 130 (2014) 384–395

egy was employed. For the scenarios 6–8b, it was only averagedover the 3 buildings aiming at the gain maximization strategy.

3.5.1.3. Part III: calculation of overall optimization variable. In part III,the optimization function is calculated as weighted sum of the sub-objective variables determined in part II using the followingformula:

Optimization Function¼ 1=ðw1 � ðAVERAGE LOAD CONSUMPTION PER DAY� Remaining Necessary Load EnergyÞ þw2 � Financial Gainþw3 � ðAVERAGE PV PRODUCTION PER DAY� PV Energy RemainingÞÞ ð1Þ

The variables AVERAGE_LOAD_CONSUMPTION_PER_DAY andAVERAGE_PV_PRODUCTION_PER_DAY both have the value16 kW h. The weights w1, w2, and w3 correspond to the priorityof the sub-objective as assigned in Table 3. For the simulations, asetting of the weights to the following values proved to yieldwell-performing results: w1 = 1000, w2 = 100, w3 = 1.

3.5.2. Optimal action order search3.5.2.1. Simulated annealing. To calculate an optimal set of actionorders for the three different grid prices, a modified simulatedannealing triple-optimizer was developed. A characteristic of thesimulated annealing algorithm is its ability to converge efficientlytoward the optimum (i.e. maximum or minimum) of a given mul-tivariable function even in large search spaces. The name tag‘‘modified’’ stems for the fact that it constitutes a modified versionof the standard simulated annealing algorithm [34,35]. The nametag ‘‘triple’’ stems from the triplication of the optimizer to findthree optimal action orders, one for each grid price.

3.5.2.1.1. Standard simulated annealing approach. Annealing is astandard process used in metallurgy to improve material proper-ties. Basically, heating makes it possible to break bonds and over-come ‘‘energy barriers’’. This enables nature to explore differentpossible structural states and find the most stable one. In furtherconsequence, this most stable state or ‘optimum state’ propagatesand gets fixed in the whole material during a controlled (i.e. slow)cooling process. In other words, via annealing, the crystalline struc-ture of the material converges towards the optimally stable state.

Table 5Values of individual optimization variables of sub-goals and overall optimization function.

ScenarioID

Total state space search Simulated annealing Optimization error interms of financial gainfor simulated annealingapproach (%)

Optimization error interms of financial gainfor simulated annealingapproach (%)

Financialgain(Units)

Notprovidedloadsupply(kW h)

PVenergywaste(kW h)

Optimizationvalue�1

Financialgain(Units)

Notprovidedloadsupply(kW h)

PVenergywaste(kW h)

Optimizationvalue�1

Gain maximization via energy exchange with main grid1 0.02 0 0 16018.1 0.02 0 0 16018.1 0.0 0.002a 9.61 0 0 16976.5 9.61 0 0 16976.5 0.0 0.002b 134.42 0 0 29458.1 134.42 0 0 29458.1 0.0 0.00

Gain maximization via energy exchange with main grid and neighbours (unified neighbour group)3 0.02 0 0 16018.1 0.02 0 0 16018.1 0.0 0.004a 9.61 0 0 16976.5 9.60 0 0 16976.0 0.1 0.004b 134.42 0 0 29458.1 134.42 0 0 29458.1 0.0 0.005a 9.82 0 0 16998.2 9.78 0 0 16993.8 0.4 0.035b 134.42 0 0 29458.1 134.42 0 0 29458.1 0.0 0.00

Gain maximization via energy exchange with main grid and neighbours (mixed neighbour group)6 4.06 0 0 16421.8 4.06 0 0 16421.8 0.0 0.007a 11.85 0 0 17200.6 11.85 0 0 17200.6 0.0 0.007b 138.47 0 0 29863.3 138.47 0 0 29863.3 0.0 0.008a 12.77 0 0 17292.9 12.52 0 0 17268.1 2.0 0.148b 142.44 0 0 30260.3 141.67 0 0 30182.7 0.5 0.26

R. Velik, P. Nicolay / Applied Energy 130 (2014) 384–395 391

The simulated annealing (SA) algorithm mimics this behaviourto find the optimum of a multivariable function. In a first step, itexplores solutions distributed over the whole search space. Aftersome time, the algorithm progressively reduces its explorationrange and starts exploring solutions in more circumvented regions.One particularity of the simulated annealing algorithm is thatthroughout the whole optimization process, it accepts from timeto time worse solutions as new reference solution. This signifi-cantly increases the likelihood of finding the globally optimal solu-tion of the function by reducing the probability of getting blockedin local optima. The decision about accepting or declining a worsesolution is based on the Metropolis criterion [34], which depends(1) on a virtual ‘temperature’, which steadily decreases throughoutthe optimization process, and (2) the deviation Df = (fnew � fref) ofthe worse solution fnew from the current reference solution fref.The higher the temperature and the smaller the deviation, themore likely is the acceptance of a worse solution. As opposed tothis, lower temperatures and larger deviations decrease the proba-bility of acceptance.

At algorithm start, the variables of the function to be optimizedare initialized with random values and the corresponding solutionis calculated, which serves as first reference solution. For all subse-quent iterations, the values of the function variables are alteredwithin a certain range of the latest reference solution. For this pur-pose, the SA algorithm usually uses a Gaussian random numbergenerator (GRNG), which generates values that are added to thereference variable solutions to obtain new values. The standarddeviation (SD) of the GRNG is adjusted beforehand according tothe desired exploration range. The new set of variable values isthen used to compute a solution of the optimization function,which is compared to the best one so far (i.e. the first one whendealing with the second iteration). If the new solution is betterthan this reference solution, it is automatically accepted as thenew reference and therefore as the new starting point for the sub-sequent exploration of the solution space. If the new solution isworse, the Metropolis criterion is used to determine whether thenew solution is accepted as the new reference and a new cyclebegins. The optimization usually stops when the solution doesnot improve further after a certain number of trials. A moredetailed description of SA algorithms can be found in [34–36].

3.5.2.1.2. Modified simulated annealing triple optimizer. The SA algo-rithm is normally used to optimize a function of continuous vari-ables. As we demonstrated in previous work [37], it is howeverpossible to modify the algorithm to optimize the ranking of a setof rules. This is done by assigning to each rule a position withina continuous number scale and correlating the relative positionof the rules to their ranking. For instance, if the range of the num-ber scale is between 1 and 10 and 3 rules have to be ranked, rule 1might initially take the position 2.158, rule 2 the position 3.984,and rule 3 the position 1.252. Accordingly, their ranking R = R1

would be 2-1-3. This ranking can then be used to calculate F(R1),where F is a given optimization function. To find the optimum ofF, the SA algorithm can alter the continuous position variables toyield the optimized ranking Ropt.

The first main difference of this ‘‘modified’’ SA algorithm incomparison to the standard SA method is that the result of F(R)changes only when rules swap their relative positions. Forinstance, if the rules 1, 2, and 3 have the respective positions5.123, 7.852, 2.128, the ranking (R2) is also 2-1-3. This means thatF(R2) = F(R1) despite the fact that the values of the variables arecompletely different. To ensure that the SA algorithm keeps explor-ing solutions over the whole search space (at least at the beginningof the optimization process) a frequent swapping of ranked rulepositions should occur. This is achieved by adequately adjustingthe range of the number scale and the SD of the GRNG prior tothe optimization. Based on a trial and error procedure conductedover a great number of test optimizations involving different num-bers of actions, in [37], we empirically determined a SD of 1.5 asgood value for optimization processes as addressed in our currentarticle. The range of the number scale was chosen to equal themaximum number of rules that had to be ranked. As at maximum14 rules had to be considered, it was set to the range (0–14).

The second difference of the modified SA algorithm is that twovariables cannot have the same value (i.e. two rules cannot havethe same ranking). This issue can be solved by checking the posi-tion values and recalculating them in case of equality.

In Velik and Nicolay [37], this modified simulated annealing(MSA) procedure was used to find the best possible ranking of amaximum of 14 rules to achieve renewable energy consumptionoptimization within a microgrid. The optimization objective was

Table 6Action order leading to an optimal result.

Scenario ID Grid price Optimized action orders in dependence on grid price

Total state space search Simulated annealing

Gain maximization via energy exchange with main grid1 1 5 2 6 1 5 2 6 1

2 6 5 1 X 6 5 1 X3 5 6 1 X 5 6 1 X

2a 1 3 6 2 4 5 1 3 6 2 4 5 12 3 5 2 4 6 1 3 5 2 4 6 13 4 6 2 5 1 X 4 6 2 5 1 X

2b 1 3 5 4 13 6 2 14 1 3 5 4 13 6 2 14 12 14 5 3 13 6 1 X X 14 5 3 13 6 1 X X3 3 13 6 5 1 X X X 3 13 6 5 1 X X X

Gain maximization via energy exchange with main grid and neighbours (unified neighbour group)3 1 7 6 5 8 2 X 7 6 5 8 2 X

2 7 6 2 5 1 X 7 6 2 5 1 X3 6 2 8 5 1 X 6 2 8 5 1 X

4a 1 3 8 6 4 7 5 1 X 3 8 6 4 7 5 1 X

2 7 3 5 4 6 8 2 1 7 4 6 3 5 8 2 1

3 5 3 4 2 8 6 1 X 5 3 4 2 8 6 1 X

4b 1 6 4 5 13 14 3 7 2 1 8 6 4 5 13 14 3 7 2 1 82 5 8 14 3 4 6 7 2 13 1 5 8 14 3 4 6 7 2 13 13 2 6 14 13 5 3 1 7 8 4 2 6 14 13 5 3 1 7 8 4

5a 1 9 3 10 2 8 6 5 7 1 X X X 3 6 5 4 12 9 11 8 10 7 2 1

2 2 8 7 4 6 12 10 3 5 9 1 X 10 8 3 7 2 4 6 5 9 1 X X

3 4 12 5 11 2 8 6 1 X X X X 5 2 8 7 10 4 12 9 3 11 6 1

5b 1 8 2 6 14 9 3 4 12 5 10 7 11 1 X X 8 2 6 14 9 3 4 12 5 10 7 11 1 X X2 7 6 10 14 13 3 1 X X X X X X X 7 6 10 14 13 3 1 X X X X X X X3 9 14 4 11 5 10 7 3 13 12 2 8 6 1 9 14 4 11 5 10 7 3 13 12 2 8 6 1

Gain maximization via energy exchange with main grid and neighbours (mixed neighbour group)6 1 6 8 2 5 7 1 6 8 2 5 7 1

2 7 8 6 2 5 1 7 8 6 2 5 13 5 8 7 2 6 1 5 8 7 2 6 1

7a 1 7 3 2 6 5 8 1 4 7 3 2 6 5 8 1 42 5 8 2 4 3 6 7 1 5 8 2 4 3 6 7 13 8 4 5 6 3 7 1 2 8 4 5 6 3 7 1 2

7b 1 2 5 14 6 4 1 X X X X 2 5 14 6 4 1 X X X X2 8 5 6 7 3 4 14 1 X X 8 5 6 7 3 4 14 1 X X3 8 14 7 4 5 3 13 1 X X 8 14 7 4 5 3 13 1 X X

8a 1 6 8 11 4 3 10 2 9 7 5 12 1 6 3 5 1 X X X X X X X X

2 5 4 8 3 6 1 X X X X X X 8 4 5 6 3 7 11 12 10 9 1

3 8 4 5 6 3 9 7 12 11 10 1 X 8 4 11 12 10 7 6 2 5 9 3 1

8b 1 8 10 6 12 9 13 14 4 7 5 2 11 3 1 8 10 6 12 9 13 14 4 7 5 2 11 3 1

2 10 3 5 11 9 8 6 12 14 13 2 7 4 1 10 3 5 11 9 8 6 12 14 13 2 7 4 1

3 8 5 10 2 12 9 3 4 6 7 13 11 1 X 8 5 10 2 12 9 3 4 6 7 13 11 1 X

392 R. Velik, P. Nicolay / Applied Energy 130 (2014) 384–395

independent from the current grid price (GP). Unlike in this case,the present study focuses on financial gain maximization of inter-connected microgrids in variable GP scenarios in which the gridprice can assume the discrete values 1, 2, and 3 and the optimalranking of rules depends on the grid price (i.e. it is not the samefor GP = 1, GP = 2, and GP = 3). As the MSA developed in Velikand Nicolay [37] could not cope with changing grid prices, it hadto be ‘‘upgraded’’ to allow for parallel optimization of rule rank-ings. The result was a MSA triple optimizer (MSATO) running threeMSAs in parallel to become capable of determining three sets ofranked rules corresponding to the three different GPs. In this con-text, it has to be mentioned that, even if rule rankings for differentgrid prices run in parallel, they are not independent from eachother. Due to the use of storage elements, actions taken at a giventime instant for a given GP can have a clear influence on the actionsbest to be taken afterwards for another GP. In the optimizer, this

interdependence is considered via the calculation of the optimiza-tion function (see Section 3.5.1).

3.5.2.2. Total state space search. To validate the results of the mod-ified simulated annealing triple-optimizer, it was decided to com-pare it against a total state space search approach. The principle ofthe total state space search approach is to find the most optimalaction order by simply trying out all possible permutations ofaction combinations and calculating the corresponding optimiza-tion function as described in Section 3.5.1. Such a total state spacesearch can be easily implemented using Alexander Bogomolyn’sunordered recursive permutation algorithm [38]. However, for sce-narios with larger sets of possible actions, searching through allpossible states without further search space limitation measuresturned out to become computationally too expensive. For a set ofx actions, the resulting computational effort is (x!)3 (see Table 7).

Table 7Comparison of computational effort of total state space search and simulated annealing optimization.

Scenario ID Number of selectable actions Number of iterations

Total state space search Simulated annealing

Individual gain maximization1 4 13824 8079242a 6 373248000 12114002b 8 6.5548E+13 1,616,400

Neighbourhood gain maximization (unified neighbourhood group)3 6 373,248,000 1,211,5264a 8 6.5548E+13 1,614,5284b 10 4.7785E+19 2,020,5005a 12 1.099E+26 2,421,3965b 14 6.6256E+32 2,828,700

Neighbourhood gain maximization (mixed neighbourhood group)6 6 373,248,000 1,210,7347a 8 6.5548E+13 1,613,5447b 10 4.7785E+19 2,020,5008a 12 1.099E+26 2,416,7528b 14 6.6256E+32 2,828,658

Fig. 7. Comparison of the total state space search and simulated annealingalgorithm in terms of achieved financial gain for different test scenarios.

Fig. 8. Comparison of computing effort of total state space search and simulatedannealing algorithm depending on the number of selectable actions (logarithmicscale).

R. Velik, P. Nicolay / Applied Energy 130 (2014) 384–395 393

The 3rd power results from the fact that action rankings depend onthe grid price, which can assume 3 different values. To cope withthis computational complexity, measures had to be found toreduce the computational complexity by pre-eliminating as manysearch space areas as possible based on the integration of systemexpert knowledge and a number of pre-simulations. This includedfor instance rules like to not try out permutations that comprisedan alteration of

� actions after the action ‘‘Do No More Activity’’ had been called,� actions that aimed at directing PV energy to some sink after all

available PV energy had already been sold to the grid,� actions that aimed at supplying the loads from any source after

all necessary load energy was already covered by the grid,� actions that aim at charging or discharging the storage when it

is already full/empty,� etc.

By these adaptations, the computation time for finding an opti-mal solution could be reduced to several months when distributingcomputational load over several machines. It is obvious that aneffort as high as taken here in terms of both (1) computation timeand (2) elaboration of search space size reduction strategies is nopracticable approach for many applications and problem domains.This underlines the importance of finding alternative approacheslike the MSATO approach.

4. Results and discussion

In this section, the results of the modified simulated annealingtriple-optimizer are presented and compared against the totalstate space reference methods in terms of obtained optimizationresults (see Section 4.1) and corresponding computing effort (seeSection 4.2).

4.1. Optimized action orders and optimization values

In Tables 5 and 6, the results of the two implemented optimiz-ers are summarized. In Table 5, the obtained values of the overalloptimization function and its sub-variables are presented as wellas the optimization error of the simulated annealing approach interms of gain maximization and the overall optimization function.In Table 6, the triple-set of action orders is listed which led to thesesolutions. The number assignment of the actions correspond to theaction IDs from Table 1. Actions prioritized after action 1 (‘‘Do NoMore Activity’’) are listed with an ‘X’ as they are irrelevant and willnot be carried out. In fact, it turned out during simulation that forall investigated scenarios, more than one triple-action-set couldlead to the (globally) optimal result. However, due to limitationsin space, always only one such optimal triple-set is presented inTable 6. In case that both optimizers found the global optimum,the same action order set is listed for both optimizer approachesand the listed action sets are additionally underlined to indicatedthat a global optimum was found.

394 R. Velik, P. Nicolay / Applied Energy 130 (2014) 384–395

Analyzing obtained optimized triple-action sets of the differenttest scenarios in detail revealed several globally valid energy trad-ing rules for the system. For instance, in scenarios where a storageof energy was possible, energy was stored when the grid price waslow (1 unit/kW h). This stored energy was then used or sold whenthe grid price was high (3 units/kW h). In the scenarios 4b and 5b,which featured a charging and discharging of the own storage viaenergy exchange with the grid, the option of energy exchange withneighbours brought no additional advantage in terms of achievedfinancial gain in comparison to scenario 2b, where no neighbourenergy exchange was supported. On the other hand, energyexchange with neighbours proved to be particularly advantageousin scenarios with a mixed neighbour group consisting partly of‘‘financial gain optimizers’’ and partly of ‘‘local PV energy self-con-sumption optimizers’’ (scenario 6–8b). Here, in case of a currentlyhigh grid price, the ‘‘financial gain optimizers’’ could benefit fromthe ‘‘local PV energy self-consumption optimizer’’ group’s prefer-ence to sell their surplus PV and stored energy to neighbours fora lower price (1 unit/kW h and 2 units/kW h) instead of selling itto the grid for 3 units/kW h as this optimizes their objective ofincreasing the local consumption of locally produced PV energy.This result demonstrates that the participation of microgrid oper-ators in microgrid trading groups with divergent objectives canbe quite beneficial for the different parties.

Based on the optimization results presented in Table 5, Fig. 7graphically compares the achieved financial gain in the differenttest scenarios for the two different investigated optimizer types.As can be seen from Fig. 7, these algorithms achieved equal resultsin 9 scenarios and only slightly deviating result in 4 scenarios. Inthe scenarios 4a, 5a, and 8b, the deviation of the simulated anneal-ing approach from the global optimum was 0.1%, 0.4%, and 0.5%.The only slightly larger deviation occurred in scenario 8a, wherethe global optimization error for the simulated annealing approachwas 2%. Averaged over all scenarios, the mean error of the simu-lated annealing approach in terms of achieved financial gain was0.2% in comparison to the total state space search referencemethod.

4.2. Computing effort

In Table 7 and Fig. 8, the computing effort (number of iterationto find the optimal solution) of the simulated annealing and thetotal state space search algorithm is illustrated for the differentinvestigated scenarios. As can be seen, for the total state spacesearch algorithm, the computing effort for a number of x selectableactions is proportional to (x!)3, which leads to an ‘‘explosion’’ ofthe computation time for larger action sets. In contrast to this,the computing effort for the simulated annealing approachremains in quite circumvented boundaries and increases ratherlinearly with the number of selectable actions. As described in Sec-tion 3.5.2.1, for the scenarios with larger action sets, a completeexecution of the total state space search algorithm remained nolonger a feasible solution due to a too long processing time. Toresolve this issue, extensive pre-simulations had to be performedand additional expert knowledge had to be brought into the sys-tem to limit the number of combinations that had to be investi-gated. Accordingly, it could be demonstrated that the simulatedannealing approach is by far superior in terms of processing timein comparison to this reference method while finding solutionsequal or very close to the global system optimum.

5. Conclusion and outlook

In this article, a modified simulated annealing triple-optimizerwas presented to search for an optimal energy management

strategy in terms of financial gain maximization in a microgird,which was connected to several neighbouring microgrids as wellas to a main grid featuring variable grid prices. The performanceof the optimizer was evaluated by comparing its results to the out-comes obtained from a computationally highly expensive totalstate space search approach capable of finding a globally optimalsolution. In nine out of thirteen investigated test scenarios, thesimulated annealing triple-optimizer was able to find the globallyoptimal solution. For the four remaining scenarios, the optimiza-tion error in terms of achieved financial gain was 0.1%, 0.4%, 0.5%and 2%. Moreover, the simulated annealing approach presentedstriking advantages in terms of computing time in relation to thetotal state space search approach. While the computation effortof the total state space search approach increased by (x!)3 withx, the number of selectable action, the computation effort for themodified simulated annealing triple-optimizer increased only line-arly with x. Accordingly, it was demonstrated that a modified sim-ulated annealing approach is a powerful method for determiningglobally optimal or close to globally optimal energy managementstrategies for financial gain maximization in microgrids. The nextstep planned is now to apply this approach to further existing gridprice models (e.g., time of use pricing, variable peak pricing, real-time pricing) and investigate and compare their potential in termsof financial gain optimization for different economic applicationscenarios.

Acknowledgements

The work reported in this article has been co-funded by theEuropean Commission within the INTERREG Program for support-ing small and medium sized companies (SME) in Italy and Carin-thia (Austria) and the Austrian Research Promotion Agency (FFG)within the program Fit4Set (project Vision Step I).

References

[1] Sober A, Elmenreich W. Smart microgrids: overview and outlook, ITGINFORMATIK 2012, Workshop on Smart Grids, September, 2012.

[2] Velik R. Cognitive architectures as building energy management system forfuture renewable energy scenarios – a work in progress report. IJSEI Int J SciEng Invest 2013;2(17):68–72.

[3] Velik R, Zucker G, Dietrich D. Towards automation 2.0: a neuro-cognitivemodel for environment recognition, decision-making, and action execution.EURASIP J Embed Syst Special Issue Networked Embed Syst Energy ManageBuild 2011;2011:11.

[4] Farhangi H. The path of the smart grid. IEEE Power Energy Magazine2010;8(1):18–28.

[5] Mohn T, Piasecki R. A smarter grid enables communal microgirds. IEEE GreenTechnologies Conference; 2011. p. 1–6.

[6] Matamoros J, Gregoratti D, Dohler M. Microgrids energy trading in islandingmode. In: IEEE third international conference on smart grid communication;2012. p. 49–54.

[7] Costa L, Kariniotaki G. A stochastic dynamic programming model for optimaluse of local energy resources in a market environment. IEEE Lausanne PowerTech 2007:449–54.

[8] Basu A, Panigrahi T, Chowdhury S, Chowdhury S, Chakraborty N, Sinha A, SongY. Key energy management issues of setting market clearing price (MCP) inmicro-grid scenario. In: 42nd International universities power engineeringconference; 2007. p. 854–60.

[9] Azofra D, Martínez E, Jiménez E, Blanco J, Saenz-Díez J. Comparison of theinfluence of biomass, solar-thermal and small hydraulic power on the Spanishelectricity prices by means of artificial intelligence techniques. Appl EnergyMay 2014;121:28–37.

[10] Livengood D, Larson R. The energy box: locally automated optimal control ofresidential electricity usage. Service Sci 2009;1(1):1–16.

[11] Velik R. The influence of battery storage size on photovoltaics energy self-consumption for grid-connected residential buildings. IJARER Int J Adv RenewEnergy Res 2013;2(6).

[12] Velik R. Battery storage versus neighbourhood energy exchange to maximizelocal photovoltaics energy consumption in grid-connected residentialneighbourhoods. IJARER Int J Adv Renew Energy Res 2013;2(6).

[13] Velik R. Renewable energy self-consumption versus financial gainmaximization strategies in grid-connected residential buildings in a variablegrid price scenario. IJARER Int J Adv Renew Energy Res 2013;2(6).

R. Velik, P. Nicolay / Applied Energy 130 (2014) 384–395 395

[14] Velik R, Nicolay P. Energy management in storage-augmented, grid-connectedprosumer buildings and neighbourhoods using a modified simulatedannealing optimization, submitted, 2014.

[15] Nguyen A, Reiter S, Rigo P. A review on simulation-based optimizationmethods applied to building performance analysis. Appl Energy January 2014;113:1043–58.

[16] Manfren M, Caputo P, Costa G. Paradigm shift in urban energy systems throughdistributed generation: methods and models. Appl Energy 2011;88(4):1032–48.

[17] Parisio A, Rikos E, Tzamalis G, Glielmo L. Use of model predictive control forexperimental microgrid optimization. Appl Energy February 2014;115:37–46.

[18] Marzband M, Sumper A, Ruiz-Álvarez A, Domínguez-García J, Tomoiaga B.Experimental evaluation of a real time energy management system for stand-alone microgrids in day-ahead markets. Appl Energy June 2013;106:365–76.

[19] Marzband M, Ghadimi M, Sumper A, Domínguez-García J. Experimentalvalidation of a real-time energy management system using multi-periodgravitational search algorithm for microgrids in islanded mode. Appl EnergySeptember 2014;128:164–74.

[20] Chen Y, Lu S, Chang Y, Lee T, Hu M. Economic analysis and optimal energymanagement models for microgrid systems: a case study in Taiwan. ApplEnergy March 2013;103:145–54.

[21] Gregoratti D, Matamoros J. Distributed convex optimization of energy flows:the two-microgrid case. In: 1st International black sea conference oncommunication and networking 2013. p. 201–205.

[22] Celli G, Pilo F, Pisano G, Soma G. Optimal participation of a microgrid to theenergy market with an intelligent EMS. In: 7th International powerengineering conference 2005. p. 663–8.

[23] Mashhour E, Moghaddas-Tafreshi S. Integration of distributed energyresources into low voltage grid: a market-based multiperiod optimizationmodel. Electr Power Syst Res 2010;80(4):473–80.

[24] Kriett P, Salani M. Optimal control of a residential microgrid. Energy 2012;42(1):321–30.

[25] Mohamed F, Koivo H. Multiobjective optimization using modified game theoryfor online management of microgrid. Euro Trans Electr Power 2011;21:839–54.

[26] Chaouachi A, Kamel R, Andoulsi R, Nagasaka K. Multiobjective intelligentenergy management for a microgrid. IEEE Trans Ind Electron2012;60(4):1688–99.

[27] Zhao B, Zhang X, Li P, Wang K, Xue M, Wang C. Optimal sizing, operatingstrategy and operational experience of a stand-alone microgrid on DongfushanIsland. Appl Energy January 2014;113:1656–66.

[28] Niknam T, Azizipanah-Abarghooee R, Narimani M. An efficient scenario-basedstochastic programming framework for multi-objective optimal micro-gridoperation. Appl Energy November 2012;99:455–70.

[29] Constantopoulos P, Schweppe F, Larson R. Estia: a real-time consumer controlscheme for space conditioning usage under spot electricity pricing. ComputOper Res 1991;18(8):751–65.

[30] Houwing M, Negenborn R, Heijnen P, De Schutter B, Hellendoorn H. Least-costmodel predictive control of residential energy resources when applying lCHP.In: Proceedings of power tech 2007, Lausanne, 6p., July 2007.

[31] Xiong G, Shi D, Duan X. Multi-strategy ensemble biogeography-basedoptimization for economic dispatch problems. Appl Energy November 2013;111:801–11.

[32] Velik R, Schmid S, Rittsteiger W, Karitnig A, Wilhelmer D, Smole E. The smartenergy demo project Vision Step I – Smart City Villach. In: 11th Austrianphotovoltaic conference; 2013.

[33] Velik R, Kafka K, Neumaier L, Schmid J, Pairitsch H, Egger W, et al. Design of aPV-supplied, grid-connected storage test bed for flexibly modeling futureenergy scenarios. Smart Grids Week Salzburg 2013.

[34] Bertsimas D, Tsitsiklis J. Simulated annealing. Stat Sci 1993;8(1):10–5.[35] Yang X. Engineering optimization: an introduction with metaheuristic

applications. USA: John Wiley & Sons; 2010.[36] Press W, Teukolsky S, Vetterling W, Flannery B. Numerical recipes: the art of

scientific computing. 3rd Ed. New York: Cambridge University Press; 2007.[37] Velik R, Nicolay P. Energy management in storage-augmented, grid-connected

prosumer buildings and neighbourhoods using a modified simulatedannealing optimization, submitted 2014.

[38] Gao S, Chabini N, Al-Khalili D. Dynamic programming addition optimizationapproach for large size multipliers in FPGAs. In: 53rd IEEE internationalmidwest symposium on circuits and systems; 2010.