iDR: Consumer and Grid Friendly Demand ResponseSystem∗

Vikas Chandan#1, Tanuja Ganu#1, Tri Kurniawan Wijaya#2†, Marilena Minou#3,George Stamoulis#3, George Thanos#3, Deva P. Seetharam#1

#1IBM Research India, #2EPFL, Switzerland, #3Athens University of Economics and Business, Greece

ABSTRACTPeak demand is a major challenge for power utilities acrossthe world. Demand Response (DR) is considered to be ef-fective in addressing peak demand by altering consumptionof end consumers, so as to match supply capability. How-ever, an efficient DR system needs to respect end consumerconvenience and understand their propensity of participat-ing in a particular DR event, while altering the consumerdemand. Understanding such preferences is non-trivial dueto the large-scale and variability of consumers and the in-frastructure changes required for collecting essential (smartmeter and/or appliance specific) data.In this paper, we propose an inclusive DR system, iDR,

that helps electricity provider to design an effective demandresponse event by analyzing its consumers’ house-level con-sumption (smart meter) data and external context (weatherconditions, seasonality etc.) data. iDR combines analyticsand optimization to determine optimal power consumptionschedules that satisfy an electricity provider’s DR objectives- such as reduction in peak load - while minimizing the in-convenience caused to consumers associated with alterationin their consumption patterns. iDR uses a novel context-specific approach for determining end consumer baseline anduser convenience models. Using these consumer specificmodels and past DR experience, iDR optimization engineidentifies -(i) when to execute a DR event, (ii) who are theconsumers to be targeted for the DR, (iii) what signals to besent etc. Some of iDR’s capabilities are demonstrated usingreal-world house-level (smart meter) as well as appliance-level data.

Categories and Subject DescriptorsH.4 [Information Systems Applications]: Miscellaneous

∗Supported by European Union’s Seventh Framework Pro-gramme (FP7/2007-2013) 288322, WATTALYST.† The work is done during the authorSs internship at IBMResearch, India

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.e-Energy’14, June 11–13, 2014, Cambridge, UK.Copyright 2014 ACM 00-00 ...$15.00.

General TermsUser preferences, Human Factors, Optimal Scheduling

KeywordsDemand Response, Energy Consumption Scheduling, UserPreferences, Smart Grids, Smart Meters

1. INTRODUCTIONDemand Response (DR) programs aim to alleviate the

peak demand problem and to provide higher system reliabil-ity by altering consumer demand in response to power grid’ssupply and economic conditions. Various evaluation studiesindicate that DR can be an effective mechanism for address-ing the challenges of growing energy demand and relatedsupply-demand imbalances [1–3]. A staff report by FederalEnergy Regulatory Commission [4] estimates that the feasi-bility of peak demand reduction in the United States can beup to 20% by using DR technologies with full participation.As per another study [5], DR programs alone could achieveup to half of the European Union’s 2020 targets concern-ing energy savings and CO2 emissions. Additionally, manyelectricity suppliers around the world are deploying smartmeters to gather fine-grained spatio-temporal consumptiondata, and to provide a bi-directional communication mech-anism [6], as shown in Figure 1. This infrastructural en-hancement is a significant enabler for bringing DR vision toreality.

However, the success of such DR programs essentially de-pends upon the end consumers’ participation. Various eval-uation studies [7, 8] based on results of DR pilots and sur-veys of participants indicate that discomfort/inconveniencecaused during a DR event is a key factor that adverselyaffects DR participation. Additionally, it was found thatdemographic attributes such as family size, income level,participants’ activity structure, etc. are relevant factors forDR participation. Hence, any effective DR system needs tounderstand end consumers’ electricity usage preferences andtheir demographic attributes while choosing the right set ofconsumers for a DR contract and/or for a particular DRevent.

Furthermore, energy suppliers/aggregators offer varioustypes of DR contracts with the end consumers, such as Timeof Use (ToU) Rates, Capacity Biding Programs (CBP), De-mand Biding Programs (DBP) and Peak Day Pricing (PDP)[9]. Any particular DR program needs to confer fairnessamongst consumers under the same DR contract. Addi-tionally, it should also take into account adverse side-effects

Figure 1: Smart metering projects across the world. AMR(Automatic Meter reading) is the technology of automati-cally collecting consumption information from energy me-ters, where as AMI(Advanced Metering Infrastructure) isthe technology that provides high-resolution consumptioninformation and provides bi-directional communication be-tween meter and utility.

of DR, such as rebound effect [10]. Considering above men-tioned challenges, it is non-trivial to determine - (i) the timewindow for executing a DR event, considering supply con-ditions and predicted consumer demand, (ii) the right set ofconsumers to be targeted for a particular DR event consider-ing factors such as end consumers’ preferences, DR contractswith these consumers and their past response to DR eventsunder similar context (like weather conditions, day of weeketc.), (iii) expected reduction per consumer and expected DRoutcome.In this paper, we present iDR, an inclusive DR plan-

ning system, that helps energy suppliers design effective DRevents. Our objective is to enable energy suppliers to designDR events while taking into account consumers’ preferencesand fairness of the system with respect to the DR contracts.The key contributions of this paper are:

1. A novel context-based method to determine baselineconsumption and quantify inconvenience caused to con-sumers (in the form of utility functions), smart meterdata.

2. A novel and simple context-based approach to deter-mine inconvenience (in the form of utility functions)using appliance-level data, if available.

3. A stochastic optimization framework that uses the abovementioned utility functions to determine which con-sumers to be targeted for DR and what signals to besent.

4. A mechanism to ensure fairness amongst multiple con-sumers with same type of DR contract while planningDR events.

The remainder of this paper is organized as follows: InSection II, we provide an overview of the related work. InSection III, we describe our approaches for determining endconsumer preferences of electricity usage depending uponthe time of day and external context etc. In Section IV,we define our optimization framework for optimal and fairDR scheduling considering supply conditions, end consumer

preferences, past DR experiments etc. We showcase the ex-perimental evaluation of iDR on some real-world datasets inSection V, and finally, in Section VI, we conclude our work.

2. RELATED WORKA body of work on autonomous DR exists in literature.

In [11], the authors propose a system, Yupik that gener-ates appliance usage schedules to minimize both a house-hold’s power bill and potential lifestyle disruptions duringa DR event. [12] presents a methodology to schedule ap-pliances taking into account cost minimization, schedulingpreferences and climactic comfort requirements. In [13], theauthors propose green scheduling as an approach to reducethe peak demand of a large number of heating systems. Atwo-stage optimization scheme is presented in [14] for HVACdemand response that analyzes the effectiveness of selectedDR strategies through on-line simulation. An autonomousDR mechanism is proposed in [15] where electrical devicescommunicate among themselves to organize autonomouslyinto control groups and to negotiate the most appropriateform of collective DR adoption. Our work complements andsupports these efforts by providing a system that can be usedby a utility provider to plan DR events. This enables theuse of autonomous DR systems, such as the ones proposed inthe above body of work, for providing load scheduling toolsto consumers to react to such DR events appropriately.

Optimal DR based on utility maximization was proposedearlier in [16], where the dual objectives are formulated forcost minimization from the utility provider’s perspective,and social welfare maximization from the consumers’ per-spective. [17] extends this work for computation time reduc-tion by proposing load consolidation and a LP framework.While these papers attempt to achieve DR objectives bothfor the utility provider and for the consumers, their practi-cal usefulness is somewhat limited because of the underlyingcomplexity associated with the use of appliance level util-ity models only. For example, a utility provider might nothave access to appliance level consumption information forall its consumers. This motivates the need for a DR plan-ning methodology which is based on smart meter data aloneand/or appliance-level data, if available.

Recently, smart meter data has been used for other as-pects of DR such as consumer segmentation [18, 19]. How-ever, this data is not used for utility mining and subsequentDR planning. The proposed work addresses this gap by pro-viding a methodology to plan and schedule DR events basedon smart meter data alone as well as combining appliance-level data if available.

Lastly, a body of literature also exists on baseline con-sumption estimation. KEMA Inc. [20] and EnerNOC [21]provide an excellent overview of the demand response (DR)baseline estimation methods employed by utilities in theUnited States. Although for simplicity, the simulation ex-periments in this paper use the mean consumption in a con-text to estimate the baseline, any other baseline estimationmethod presented in these references (e.g. HighXofY or Ex-ponential Moving Average) can be similarly used.

3. DETERMINING USER PREFERENCESIn this section, we present methodologies to model the

utility derived by a consumer as a function of her consump-tion. These models form the basis of the optimization frame-work presented in section 4. We propose two utility mod-

eling frameworks - one at the aggregate house level (usingsmart meter data), and another which uses appliance levelconsumption data if such high resolution sensing infrastruc-ture is available.

3.1 Preference mining using smart meter dataAnalysis of historical consumption data for a consumer

over time can provide information on her consumption pref-erences. We use two quantities - baseline and utility - toquantify such preferences. Baseline is defined as an estimateof the electrical load drawn by a consumer in the absenceof any DR related curtailment actions. Utility is definedas the ‘benefit’ obtained by a consumer corresponding to agiven amount of consumption. The objective of preferencemining in this work using smart meter data is to determinebaselines and utilities for each consumer. The underlyingmethods are explained below.

3.1.1 Baseline estimationExisting methods for baseline estimation involve averag-

ing [20], regression [22] and time series analysis [23]. In thiswork, we obtain baselines by first identifying appropriateconsumer contexts. A context is defined as a combinationof external and internal factors that influence a consumer’sconsumption decisions. For instance, at a month level, influ-encing contexts could be season of a year (summer, autumn,winter or spring) or at a finer resolution month of the year(January, February, etc.). Similarly, at a day level, possiblecontexts are weekday/weekend or day of the week(Monday,Tuesday, etc.).Multiple contexts may be associated with a particular day.

As an example, October 17th, 2013 can be represented bycontexts such as {Autumn}, {Autumn, weekday}, {Octo-ber, Thursday}, etc. To choose the most appropriate con-text, and obtain the corresponding baseline consumptions,we proceed as follows:

1. We denote the day for which baselines are to be esti-mated by d. We assume that this day is divided intoT discrete timeslots. We first identify the set C(d)consisting of all contexts and their combinations thatexplains the day d.

2. Based on the historical data, we find the context C∗ ∈C(d), whose consumption has lowest dispersion. Heredispersion can be quantified using statistical measuressuch as standard deviation or inter-quartile range.

3. The baseline consumption for each timeslot t ∈ {1, 2, ..., T}is then determined by a measure of central tendency- such as mean or median - applied to historical con-sumption data at timeslot t on previous days in thecontext C∗.

In obtaining the baseline consumption as described above,historical data can be used in several different ways. Thesimplest method is to use a static time window, where con-sumption data from all previous days in the context are used,irrespective of how far in the past these days are from the dayd. On the other hand, a moving window approach only usesconsumption data from Nprev most recent days. An expo-nentially moving average approach is a variation of the mov-ing window approach, where a weighted average is performedwith more importance (higher weights) being given to themost recent days. For a comparative analysis of various

A B C D E F0.4

0.42

0.44

0.46

0.48

0.5

0.52

0.54

0.56

0.58

MA

E (

in k

Wh)

Residential consumers

A B C D E F80

85

90

95

100

105

110

115

120

MA

E (

in k

Wh)

Commercial and Industrial consumers

Figure 2: Evaluation of proposed baseline estimationmethod with respect to existing approaches based on av-erage mean absolute error (MAE) across all consumers forResidential consumers [24] and Commercial Consumers [25]datasets. Approaches A and B present our proposed ap-proaches - (A) Enlarge window with exponentially mov-ing average and (B) Enlarge window, approaches C to Gpresent existing methods [20] - (C) CAISO, (D) ISONE, (F)Low5Of10 and (G) NYISO.

methods for baseline prediction, the user is directed to thestudy [20]. Figure 2 provides evaluation of proposed base-line estimation method with respect to existing approachesbased on average mean absolute error (MAE) across all con-sumers for Residential consumers [24] and Commercial Con-sumers [25] datasets. Enlarge window with exponentiallymoving average method(A) performs better than existingapproaches on both datasets.

3.1.2 Utility quantificationTo express the utility derived by a consumer as a function

of her consumption, we make the following assumptions:

1. A consumer derives maximum utility when her con-sumption equals baseline, since by definition the base-line represents the preferred level of consumption.

2. Any deviation from baseline - either above or below -results in a reduction of utility.

In order to satisfy the above assumptions, we propose theuse of probability distribution of consumption during thespecific timeslot (on the past consumption data filtered byoptimal context C∗) to quantify utilities. The probabilityassociated with a consumption of q kWh, reflects the con-sumer’s preference for consuming an amount of energy q

KWh during the specified slot and under the context C∗.To make this setting more formal, we introduce the conceptof utility functions.The utility function is formally defined as follows:

Consider a consumer i, and a timeslot t in the day, wherei ∈ {1, 2, ..., N}, and t ∈ {1, 2, ..., T} and d is the day in con-sideration for DR planning. Let C∗ be the optimal contextfor the triplet {i, t, d} based on the method discussed aboveandQb

i,t,d be the corresponding estimated baseline consump-tion. For any particular consumption Qi,t,d by the consumerat this time slot of the day, associated utility Ui,t,d(.) is de-termined as:

Ui,t,d(Qi,t,d) =pC

∗(Qi,t,d)

pC∗(Qbi,t,d)

(1)

where pC∗(q) gives the probability of consumption q in the

past data under optimal context C∗.The consumer preferences - captured via baselines and

utility functions as described above - are inputs that areused to set up the optimization frameworks presented in thenext section.

3.2 Preference mining using appliance dataSo far, we have presented an approach for quantifying con-

sumer preferences using aggregated, household level, con-sumption data. Nevertheless, assuming that appliance leveldata is available, for example where AMIs are in place, thereis an evident potential to leverage this information in orderto form a more detailed and accurate depiction of user’s pref-erences. This section presents a methodology for achievingthat. The proposed methodology is based on mining theutility functions of each monitored appliance. More specifi-cally, it estimates the importance that each consumer placeson each specific appliance via calculating a corresponding“weight”. These weights constitute a measure of indicatinghow flexible (elastic) the consumers are in altering the typ-ical use of each appliance, for example in the case of a DRevent.Our methodology follows three distinct phases. At first,

by using statistical analysis, we derive the utility values ofeach appliance in order to formulate the consumption pro-file of each household. Next, we fit the derived utility valuesfrom phase 1 to those resulting from the utility functionsof [16], thus also validating the results from our approachagainst a more theoretic one that assumes the knowledge ofutility functions. Finally, exploiting the fitted utility func-tions we mine the weights that reflect consumer’s preferencesregarding the usage of appliances, under specific realistic as-sumptions.As a prerequisite, our work adopts the categorisation of

household devices that is proposed in the widely cited work[16]. According to [16], the typical household appliances canbe classified into four types; each type is characterized by autility function Ui,α(qi,α) that models how much consumeri values the consumption vector qi,α and a set of linear con-straints on qi,α.To experimentally evaluate our approach, we use real con-

sumption data from 6 households in India. The availabledata comprises of sensor readings at a granularity of ten sec-onds for four different appliances: air-conditioning, fridge,washing machine and television (TV). These reading are ex-tracted for a given context, i.e. weekdays in July and belong

to three of the appliance categories proposed by [16], as thefridge and the air-conditioner both belong to the same, Type1 category.

For conciseness, we present application details of our pro-posed methodology to only one of the three categories ofdevices (Type 4). However, the same approach was used forthe other devices, results for which are included.

3.2.1 Phase 1: Derivation of utility values from ap-pliance level data

In Phase 1 the utility values for the respective devicesare derived using statistical analysis. Let us now explainthis process for Type 4 devices. Type 4 category includesappliances such as TV, video games, and computers that acustomer uses for entertainment. In this case, the customer’sutility depends on two factors: how much power is consumedat each time she wants to use them, and how much totalpower is consumed over the entire day. At each time, t ∈{1, 2, ..., T}, we assume that the consumer i attains a utilityUi,t,α(qi,t,α) from consuming qi,t,a on appliance α.

1 3 5 7 9 11 13 15 17 19 21 230

20

40

60

80

100

120

140

Timeslot

Pow

er C

onsu

mpt

ion

Day 1Day 2Day 3Day 4Day 5Day 6Day 7Day 8Day 9Day 10Day 11Day 12Day 13Day 14Day 15Day 16Day 17Day 18Day 19

Figure 3: The per timeslot operation of type 4 appliancesduring weekdays in July

By constructing the histograms of the consumption vec-tor Qi,j,α := {qi,t,α, t ∈ {1, 2, ..., T}}, over all timeslots, asshown in Figure 3, we can analyse the consumer behavior i.e.the timeslots when their appliances are operated and the as-sociated consumption level. For this type of appliances wecan sensibly assume that higher the consumption, greateris the satisfaction experienced by the consumers, which ineconomic theory is expressed as a “utility value”. At thispoint, as we do not have any knowledge of the function thatexpresses the relationship between the consumption valuesand the obtained utility values, the latter can be approxi-mated by the empirical cumulative distribution function forgiven consumer i, appliance α, timeslot t and context andas follows:

Ui,t,α(qi,t,α) =

0, if qi,t,α < qmin

i,t,α

eCDFi,t,α(qi,t,α), if qi,t,α ∈ [qmini,t,α, q

maxi,t,α]

1, if qi,t,α > qmaxi,t,α

(2)Here, qmin

i,t,α and qmaxi,t,α are minimum and maximum con-

sumption observed in timeslot t, respectively. In particular,

given the range of observed consumption values [qmini,t,α, q

maxi,t,α],

and by assuming that the utility is an increasing function ofconsumption and thus strictly monotonic, we define the util-ity value associated with a consumption x as the cumulativefrequency of consumption valus <= x in this range.Note that, when calculating the probability of a consump-

tion value to be less than a certain value, the eCDF takesinto consideration situations of zero consumption, i.e. whenno utility is gained. Therefore, we don’t take such situationsinto account in deriving the utility values.To define the utility value Ui,j,α(Qi,j,α) for a specific day

j, a given consumer i, a given appliance α and for a givenprofile Ui,j,α, where

Qi,j,α = [qi,1,α, qi,2,α, ..., qi,24,α]T ,

we estimate the utility value of a day j as the median -weighted sum:

Ui,j,α(Qi,j,α) =

∑t(mediani,t,α ∗ Ui,t,α(qi,t,α))∑

t(mediani,t,α), (3)

where, mediani,t,α is the median of consumption data ofappliance α in timeslot t for a consumer i and given context,and it is used in order to avoid the analysis to be skewed infavour of very high or very low consumption values.

3.2.2 Phase 2: Fitting the estimated utility values tothe utility functions of [16]

In the next step, our approach utilizes the utility functionsof the different types of appliances from [16] representedas continuously differentiable concave functions of the totalconsumption in timeslot t and day j. Our objective is toapproximate these coefficients such that the resulting utilityvalues match those statistically estimated in Phase 1. Thiscan be achieved by minimizing the distance of the curveformed from the utility values, i.e. the eCDF curve at thepoints of evaluation, from the curve of the utility functionsin [16].We adopt the utility functions as well as the initialization

data defined in the experimental section of [16]. In partic-ular, for Type 4 appliances, the utility function takes theform of:

Ui,t,α(qi,t,α) = Cα − (bα − qi,t,αq

)−1.5 (4)

where Cα ≥ 0, bα ≥ 0 and q > 0 are the variables to befitted. In this case, the fitting is executed at each timeslot tdue to the nature of the appliance and corresponds to a nonlinear regression problem.Figure 4 presents the results, while the notation follows

the nomenclature in Table 1. The coefficients obtained usingleast squares fit are Cα = 2.2953, bα = 0.5786 and q =7.5397∗103. From Fig. 4, we observe that the utility values,i.e. magenta dots, after fitting are close to the values fromthe utility functions, i.e. dark green dots.In general, the results from the fitting methodology for all

appliances imply that our approach of statistically derivingthe consumers preferences, in terms of utility values, withuse of detailed consumption data and without any knowl-edge about the utility functions has small deviations fromthe case where full knowledge is available, as in [16].

3.2.3 Phase 3: Mining the weights of each applianceTo derive the weights of each appliance that reflect the

consumer’s preferences, we use the assumption that given

1 3 5 7 9 11 13 15 17 19 21 230

0.5

1

1.5

2

Timeslot i

Util

ity V

alue

eUtv,i

Utv,i

Utv,fit,i

normalized eUtv,i

normalized Utv,i

normalized Utv,fit,i

Figure 4: TV - Fitting the derived utility values to utilityvalues based on [16] for Monday 15.07.2013

eUα,t estimated utility values derived inPhase 1 for appliance α in timeslott.

Uα,t utility values resulting from theutility functions [16] for applianceα in timeslot t.

Uα,fit,t utility values resulting from apply-ing the fitted coefficient values tothe utility functions for appliance αin timeslot t.

normalized eUα,t is calculated aseUα,t

max(eUα,t)

normalized Uα,t is calculated asUα,t

max(Uα,t)

normalized eUα,fit,t is calculated asUα,fit,t

max(Uα,fit,t)

Table 1: Nomenclature of Figures

two different days of the same context, e.g. two weekdays,the consumption profile might vary, but the total Net Benefit(NB) obtained (i.e. the utility gained minus the monetarycost) is the same. This implies that for both days if thecharging scheme is known - a valid assumption in our case -each consumer chooses that consumption, which maximizesher total NB. To apply this to our methodology we divideeach day in 4 equal timeslots and calculate one respectiveweight per timeslot for the TV, fridge and air-conditionerand one per day for the washing machine. We use only oneweight for the washing machine as we are interested onlyin the total consumption of the appliance after each oper-ation, since the respective load (in one washing cycle) canbe interrupted by the consumer and shifted to subsequenttimeslots, which cannot be the case, for example, in the useof an air-conditioning device.

Thereby, given a set of appliances α ∈ A,A :={AC,FR,WM,TV }, the weights wi,α, α ∈ A and the samecontext for each day j, j ∈ J , the NB is of the followingform:

NBi,j =∑α∈A

wi,αUi,j,α(−−−→Qi,j,α)− Ci,j(

∑α∈A

−−−→Qi,j,α), (5)

where−−−→Qi,j,α = [qi,1,α, qi,2,α, qi,3,α, qi,4,α], and

Ci,j(∑

α∈A

−−−→Qi,j,α) are the cost functions based on which the

consumers are charged. Note that in this phase we considerall type 1 appliances of consumer i as a single appliance.Thus, by equating the NB formulas for all the combinationsof days from the available set of days J and applying linearleast squares optimization, we derive the weights presentedin Table 2.

Timeslot wi,Type4 wi,Type1

1 0.0002 0.000142 0.0008 0.34253 0.0157 0.55774 0.0049 0.00007

wi,Type2

0.1589

Table 2: Values of appliances’ weights using NB

The results indicate that the consumers are rather inelas-tic to changes in the consumption of type 1 appliances, whichis rational due to the fact that the use of such appliances (fore.g. cooling devices) in the given context of summer week-days is of paramount importance to them. We also observethat the weights that correspond to type 4 appliances aresmaller than those of type 1 and type 2 appliances. Thisimplies that this type of load can be curtailed at a specifictime period or shifted to another time period, without caus-ing significant inconvenience to consumers.On the other hand, we observe that type 2 appliances are

more elastic in terms of the time of their operation. Thismeans that their load cannot be curtailed or shifted before aspecific task is finished but they can operate at anytime dur-ing the day, which explains why the resulting value shown inTable 2 is greater than the values of type 4. Generally, theconclusions one can deduce about the importance of the ap-pliances in the consumers’ everyday life are consistent withthe respective bibliography about the categorization of ap-pliances in terms of shiftable and curtailment load. How-ever, someone (for e.g. an energy provider) can leverage ourmethodology to quantify each consumer’s personal prefer-ences, and use them in order to predict their behaviour inthe case of a DR signal being sent to them.Although we have presented two frameworks for under-

standing and quantifying consumers’ preferences, in the re-mainder of this paper we use the framework based on smartmeter data for further development of iDR. This is becausethe rapid penetration of smart meters in today’s grids haveallowed utility companies access to consumers’ aggregateconsumption data at the household level. Therefore, we be-lieve that at the current state of metering insfrastructure,the framework based on smart meter data is more univer-sally applicable than that based on appliance level data.However, we recommend the use of appliance level prefer-ence mining, wherever possible, as we believe it is a more ac-curate representation of consumer’s consumption patterns.

4. OPTIMIZATION FRAMEWORKIn this section, we present a methodology that can be

used by the utility provider for planning DR events. In par-ticular, for a given day, we seek to determine the followingin advance: (i) when should DR events be conducted, (ii)which consumers should be targeted, and (iii) what DR sig-nals should be sent to each such consumer. Our approach

involves solving an optimization problem, which uses con-sumers’ preferences quantified in the previous section. Thegoal of the optimization is to plan DR response events suchthat the inconvenience caused to consumers is minimizedwhile the utility provider’s targets with regard to reductionin consumption are achieved.

Before a formal presentation of the framework, we first de-fine the underlying nomenclature. The number of consumersserved by the utility is denoted by N . The day for which DRplanning is to be performed is denoted by d. It is dividedinto T timeslots. Consider a consumer i, and a timeslott in the day, where i ∈ {1, 2, ..., N}, and t ∈ {1, 2, ..., T}.For the triplet {i, t, d}, the best matching context C canbe obtained using the methodology described in Section 2.Using this context, a baseline consumption Qb

i,t,d and thecorresponding utility function Ui,t,d(.) are determined.

We assume that for each timeslot t in the day d underconsideration, the utility provider determines, in advance,the maximum amount of energy, Qs

t,d to be supplied. Sucha determination can be made based on a combination of theprovider’s power procurement schedule from power generat-ing companies for that day, and economics associated withpower purchase. For instance, if it is known that buyingelectricity during certain time periods (e.g. peak demandperiods) would be more expensive than during other peri-ods, the utility provider may choose to limit the amount ofenergy procured during those time periods.

Given the above setting, we first determine the set TDR

of timeslots when DR events should be conducted. This setconsists of timeslots when the expected (baseline) consump-tion, added across consumers exceeds the desired supply,and is more formally expressed as:

TDR =

{t :

N∑i=1

Qbi,t,d ≥ Qs

t,d, t ∈ {1, 2, ...T}

}. (6)

To determine the consumers to be targeted for DR and theDR signals to be sent, we propose the following optimizationframeworks. The first framework assumes that the targetedconsumers would always participate in the DR event. Thesecond framework assumes that these consumers would par-ticipate only with some pre-determined probability. Two de-sign variables - nt and ηt

max - are used in these frameworks.The former represents the maximum number of consumersthat the utility would like to target during timeslot t for par-ticipation in a DR event, whereas the latter represents themaximum reduction in consumption, expressed as a fractionof the baseline consumption Qb

i,t,d for each consumer, thatthe utility provider can request during a DR event at times-lot t. The use of nt as a design variable is motivated by theneed to reduce the computational requirements associatedwith the optimization. For example, in a realistic scenario,a utility provider can cater to millions of consumers, butmight like to involve only a few thousand consumers at anygiven time for DR.

4.1 Deterministic frameworkFor all t ∈ TDR, the following optimization problem is

solved:{ν∗i,t,d,∆Q∗

i,t,d

}=

argmax{νi,t,d,∆Qi,t,d}

Jd :=

N∑i=1

Ui,t,d

(Qb

i,t,d − νi,t,d∆Qi,t,d

)(7)

Subject to:

N∑i=1

(Qb

i,t,d − νi,t,d∆Qi,t,d

)≤ Qs

t,d, (8)

νi,t ∈ {0, 1} for all i ∈ {1, 2, ..., N}, (9)

N∑i=1

νi,t ≤ nt, and (10)

0 ≤ ∆Qi,t,d ≤ ηtmaxQ

bi,t,d for all i ∈ {1, 2, ..., N}. (11)

The objective of the above optimization, expressed by (7)is to maximize the aggregated utility obtained by the con-sumers during a DR event. In other words, we seek to min-imize the inconvenience caused to the consumers resultingfrom reduction in consumption. The decision variables inthe framework are as follows: (i) for each i ∈ {1, 2, ..., N},the boolean variable νi,t,d represents whether consumer ishould be targeted for DR during timeslot t, indicated bya value of 1, and 0 otherwise; (ii) for each i ∈ {1, 2, ..., N},the varible ∆Qi,t,d represents the target reduction in con-sumption for consumer i during the DR timeslot t. The con-straint (8) restricts the aggregate consumption during eachDR timeslot t ∈ TDR to be within the consumption boundQs

t,d imposed by the utility provider. The DR design vari-

ables nt and ηtmax, explained earlier, appear in constraints

(10) and (11), respectively.

4.2 Stochastic frameworkWe denote the probability of user i responding to a DR

event at timeslot t in day d by pi,t,d. This probability canbe determined, for instance, by analyzing the user’s responsehistory to previous DR signals sent to her on days and times-lots which correspond to the context C.Next, we introduce a random boolean variable αi,t,d, which

indicates the ith consumer’s decision to participate, whenthe utility provider targets that consumer for a DR event.This setting can be mathematically expressed using the fol-lowing equations:

αi,t,d ∈ {0, 1} for all i ∈ {1, 2, ..., N}, and (12)

P (αi,t,d = 1|νi,t,d = 1) = pi,t,d. (13)

A modified objective function Js is defined as shown below,where E(.) is the expected value operator.

Js := E

(N∑i=1

Ui,t,d

(Qb

i,t,d − αi,t,dνi,t,d∆Qi,t,d

)). (14)

The stochastic version of the constraint (8) is given by

E

(N∑i=1

(Qb

i,t,d − αi,t,dνi,t,d∆Qi,t,d

))≤ Qs

t,d (15)

The stochastic optimization framework, therefore seeks tomaximize Js given by (14), subject to the constraints (9)

to (11), and the constraints (12), (13) and (15). Hence itrepresents a chance-constrained mixed integer program. Theexpected values appearing in equations (14) and (15) canbe evaluated using (13), resulting in deterministic versionsof these equations given by (16) and (17), respectively. Itshould be noted that by setting pi,t,d to 1, we recover thedeterministic framework presented in Section 4.1.

Js =

N∑i=1

[Ui,t,d

(Qb

i,t,d

)− νi,t,dpi,t,dX

]where X =

{Ui,t,d

(Qb

i,t,d

)− Ui,t,d

(Qb

i,t,d −∆Qi,t,d

)}.

(16)

N∑i=1

(Qb

i,t,d − νi,t,dpi,t,d∆Qi,t,d

)≤ Qs

t,d. (17)

The use of a stochastic framework can also allow the utilityprovider to ensure fairness on the basis of DR contract. Forexample, if the contract specifies a maximum number of DRsignals Nmax

DR,i to be sent to a consumer i in a given time slott, the probabilities pi,t,d can be iteratively adjusted (e.g.once every day) as shown below:

pi,t,d = 1−Ncurr

DR,i

NmaxDR,i

(18)

Here, NcurrDR,i denotes the total number of times the con-

sumer i has been selected at time slot t since the beginningof the contract upto the current iteration. We observe that(18) results in probabilities of participation which are mono-tonically decreasing functions of the number of times theconsumer has been targeted for DR. When the consumerhas been selected Nmax

DR,i number of times, pi,t,d becomeszero, which ensures that she would not be targeted for DRany further.

4.3 Feasibility conditionIn this section, we prove a necessary and sufficient condi-

tion for feasibility of the afore-mentioned optimization frame-work, and determine a feasible point that can be used as acandidate initial point to solve it.

Theorem 1: Without loss of generality, we assume thatfor a given t ∈ TDR, the consumers indexed by i ∈ {1, 2, ..., N}are arranged in the increasing order of pi,t,dQ

bi,t,d. The prob-

lem of maximizing Js given by (16) subject to the constraints(17) and (9) to (11) admits a feasible solution if and only ifthe following inequality is satisfied:

N∑i=1

Qbi,t,d −Qs

t,d ≤ ηtmax

N∑i=N−nt+1

pi,t,dQbi,t,d. (19)

If the condition (19) is satisfied, decision variables obtainedby the assignments (20) and (21) are feasible.

If i ∈ {1, 2, ..., N − nt}, νi,t,d = 0, ∆Qi,t,d = 0, and (20)

If i ∈ {N − nt + 1, ..., N}, νi,t,d = 1, ∆Qi,t,d = ηtmaxQ

bi,t,d.(21)

Proof. We first prove the ‘necesssay’ part. If for all i ∈{1, 2, ..., N}, νi,t,d and ∆Qi,t,d are such that they satisfy (17)

and (11), they also satisfy:

N∑i=1

Qbi,t,d −Qs

t,d ≤N∑i=1

νi,t,dpi,t,d∆Qi,t,d

≤ ηtmax

N∑i=1

νi,t,dpi,t,dQbi,t,d. (22)

Since the set {1, 2, ..., N} is sorted in the increasing order ofpi,t,dQ

bi,t,d, the expression in the RHS of (22) attains a max-

imum value of ηtmax

∑Ni=N−nt+1 pi,t,dQ

bi,t,d due to the re-

maining constraints (9) and (10). Therefore, (22) can be sat-

isfied only if:∑N

i=1 Qbi,t,d−Qs

t,d ≤ ηtmax

∑Ni=N−nt+1 pi,t,dQ

bi,t,d.

To prove the ‘sufficient’ part, we assume that the condition(19) is satisfied. It can be easily verified that the decisionvariables obtained by assignments (20) and (21) satisfy thefollowing relationship for all i ∈ {1, 2, ..., N}.

νi,t,dpi,t,d∆Qi,t,d = ηtmaxνi,t,dpi,t,dQ

bi,t,d. (23)

Using (19) and (23), we establish that the above chosendecision variables also satisfy the constraint (17). Sincethese decision variables also satisfy (9), (10) and (11), weconclude that this choice of decision variables is feasible,that is it satisfies all underlying constraints. Hence the con-dition (19) leads to a feasible choice of decision variables,given by (20) and (21).

Note that if the condition (19) is not satisfied for a partic-ular choice of design parameters, either ηt

max or nt or bothmight have to be increased until it is satisfied.

5. EXPERIMENTAL EVALUATIONIn this section, we perform simulation experiments us-

ing a real world consumption data set to evaluate the DRplanning methodology presented in Section 4. We compareand evaluate performance three approaches - (1) determin-istic optimization Section 4.1, (2) stochastic optimizationpresented in Section 4.2 and (3) a rule based approach pre-sented further in Section 5.5. The goal of these experimentsis to understand and explain the results of the proposed op-timization frameworks using intuition, and to quantify theefficacy of these frameworks by comparing with the rule-based approach. Each of these experiments is performed intwo steps. In the first step, only 10 consumers are includedto facilitate an easy understanding of the results. In the sec-ond step, the optimization is repeated on the complete set ofconsumers to demonstrate the scalability of iDR. Lastly, wealso include an experiment, where we investigate the impactof the fairness strategy proposed in (18).

5.1 Data collection and pre-processingWe use consumption data available in the CER Ireland

dataset [24]. This dataset contains approximately 5000 cus-tomers, both residential and small and medium enterprises.Measurements were obtained using smart meters for a pe-riod of 1.5 years, from July 2009 to December 2010. Sincethe dataset was collected as a part of a dynamic pricingtrial, we select consumers who are in a single control group.In addition, we choose residential customers with no miss-ing data, thereby resulting in N = 500 consumers. Foreach day, and for each consumer, the recorded data is usedto obtain consumption in kWh for each one hour time slot(t ∈ {1, 2, ..., 24}) during the day. In this way, we obtain

the consumptionQi,t,d corresponding to each triplet {i, t, d},where i is the consumer index (i ∈ {1, 2, ..., N}), t representsa slot in the day (t ∈ {1, 2, ..., 24}), and d refers to a day inthe dataset.

5.2 Parameters for optimizationIn the simulation experiments reported in this section, we

choose the day dDR for DR planning as April 1, 2011, whichrepresents the first day of summer in 2011. We select theoptimal context C∗, such that dDR ∈ C∗, for each consumeras described in Section 3.1.1. Let dC be the set of daysbelonging to context C∗ for the given consumer. For thesimplicity and computational efficiency, we assume Gaus-sian distribution of consumption over the set dC . Thus, onday dDR, for each pair {i, t}, where i ∈ {1, 2, ..., N} andt ∈ {1, 2, ..., 24}, we determine the baseline consumptionQb

i,t,dDRand the utility function Ui,t,dDR(.) using equations

(24) and (25).

Qbi,t,dDR

= µi,t,dC , (24)

Ui,t,dDR (Qi,t,dDR) = exp

{−(Qi,t,dDR −Qb

i,t,dDR

)22σi,t,dC

}.

(25)Here, µi,t,dC and σi,t,dC denote the mean and standard de-

viations, respectively, of the set {Qi,t,d : d ∈ dC}. Equation(24) sets the baseline consumption for a consumer in anytimeslot to her mean consumption in that timeslot observedon days which lie in the appropriate context C∗. Equation(25) uses a Gaussian function to describe a utility value be-tween 0 and 1 for consumption in each timeslot, ensuringthat the maximum utility of 1 is obtained when consump-tion equals baseline consumption. Given the aforementionedsetup and parameters, we now describe various simulationexperiments to evaluate the proposed DR planning method-ology.

As the first step in DR planning, we determine the set oftimeslots in the day, TDR to be targeted for DR. The sum∑N

1=1 Qbi,t,dDR

of baseline consumptions for all consumers,corresponding to each hourly timeslot in the day is plottedin Figure 5. The utility provider’s assumed target energydelivery schedule is also shown in Figure 5. From a compar-ison of these plots, we determine, using (6) that the set oftimeslots to be targeted for DR is TDR = {13, 22}. To solvethe mixed integer nonlinear optimization problems (MINLP)appearing in Sections 4.1 and 4.2, we use the NOMAD al-gorithm [26] implemented in the OPTI toolbox in MAT-LAB [27]. NOMAD is a mesh-adaptive direct search algo-rithm with the potential to find global solutions to MINLPs.It should be noted that the objective of this work is not todetermine the best available solution method for the MINLPconsidered. Hence our use of NOMAD should be treated asone of the choices among several possible MINLP solvers,and the interested reader is free to use any other solverto perform these experiments. To analyze and understandthe results of the optimization, the baseline consumptionsQb

i,t,dDRand the standard deviations σi,t,dC for each con-

sumer for the DR timeslots t = 13 and t = 22 are shown inTables 3 and 4, respectively.

5.3 Approach 1: Deterministic caseIn the first aprroach, we assume that each consumer’s

5 10 15 202

4

6

8

10

12

14

Timeslot, t

Ene

rgy

(kW

h)

Sum of baseline consumptionsUtility’s target supply schedule

Figure 5: Comparison of the sum of baseline consumptionsand the utility provider’s power delivery schedule over theday dDR.

Consumernumber

Qbi,t,dDR

(kWh)σi,t,dc

(kWh)pi,t,dDR (forexperiment 2)

1 3.143 2.685 0.92 0.393 0.373 0.93 0.305 0.287 0.94 0.417 0.449 0.95 1.077 1.268 0.96 1.928 1.752 0.17 0.164 0.251 0.18 1.140 1.324 0.19 1.440 1.495 0.110 0.680 0.982 0.1

Table 3: Baseline consumptions (Qbi,t,dDR

), standard devi-ations (σi,t,dc) and probabilities of participation (pi,t,dDR)chosen for timeslot t = 13. These parameters are used torun the simulation experiments.

probability of participation in a DR event is unity, i.e. foreach t ∈ TDR, and for each i ∈ {1, 2, ...N}, pi,t,dDR = 1. TheDR design parameters nt and ηt

max which are required bythe optimization framework presented in Section 4.1 are setto 3 and 0.25 respectively. It can be easily verified that thischoice of these parameters satisfies the feasibility criterion(19) obtained earlier.

5.4 Approach 2: Stochastic caseIn the second approach, we assign non-unity probabilities

of participation pi,t,dDR in a DR event for each consumer.Since DR participation information was not reported in thedata set being used, we assign these probabilities as shownin Tables 3 and 4. With this choice of probabilities, thefeasibility condition (19) is not satisfied. To address this,we increase nt from 3 to 4.

5.5 Approach 3: Rule based strategyIn the third approach, we investigate a rule based strategy

(Algorithm 1) for determination of which consumers shouldbe targeted for DR and what DR signals should be sent. Inthe absence of appropriate ground truth information, we useresults obtained from this rule based scheme as the referenceto test the value added by the DR planning methodologypresented in Section 4. It identifies a set of nt consumers

Consumernumber

Qbi,t,dDR

(kWh)σi,t,dc

(kWh)pi,t,dDR (forexperiment 2)

1 0.675 0.357 0.92 0.988 0.626 0.93 0.729 0.491 0.94 1.060 0.962 0.95 3.116 2.261 0.96 1.079 1.062 0.17 0.388 0.594 0.18 1.701 1.304 0.19 1.723 0.966 0.110 1.275 1.153 0.1

Table 4: Baseline consumptions (Qbi,t,dDR

), standard devi-ations (σi,t,dc) and probabilities of participation (pi,t,dDR)chosen for timeslot t = 22. These parameters are used torun the simulation experiments

which achieve the targeted reduction in demand by navi-gating through the set of consumers in the increasing orderof their reduction in utility (steps 2, 3 and 4). Once a setof such consumers is obtained, it then assigns the demandreduction target for each of those consumers in such a waythat their percentage reduction over their baseline consump-tion is the same (step 5). Such an assignment is performedto ensure a ‘fair’ DR strategy across the target consumers.

Algorithm 1: Rule based strategy for DR planningat timeslot t ∈ TDR

STEP 1: For each consumer i, determine the reductionin utility ∆Umax

i,t,dDRand the absolute expected reduction in

consumption ∆Qmaxi,t,dDR

:= pi,t,dDRηtmaxQ

bi,t,dDR

from base-

line corresponding to a percentage reduction of ηtmax in con-

sumption.STEP 2: Sort consumers in ascending order of ∆Umax

i,t,dDR.

We refer to this sorted set by St. Set m = 1.STEP 3: Set the ‘current decision set’ Dt to consumers

{m,m+ 1, ...,m+ nt − 1} in St. Set m = m + 1.STEP 4: Repeat step 3 until the total expected reduction

in consumption corresponding to Dt,∑

i∈Dt∆Qmax

i,t,dDRis

greater than or equal to the targeted reduction∑N

i=1 Qbi,t,d−

Qst,d.STEP 5: The final set of consumers to be targeted is given

by Dt. The target reduction for each consumer i ∈ Dt, is

then set to: ∆Qi,t,dDR =

(∑Nj=1 Q

bj,t,d −Qs

t,d

)Qb

i,t,dDR∑j∈Dt

pj,t,dDRQbj,t,dDR

.

5.6 ResultsThe decision variables obtained using the three approaches

described above are presented and compared in Figures 6 to7. The consumers selected for DR during timeslots t = 13and t = 22, are shown in Figure 6. The corresponding DRsignals (targeted consumptions in kWh) are shown in Fig-ures 7.

Due to the choice of the utility function in (25), for thesame reduction over baseline consumption, a consumer withlarger standard deviation, σi,t,dC would be subjected to asmaller reduction in utility than a consumer with smallerstandard deviation. Also, for the same percentage reduc-tion over baseline consumption, consumers with higher base-

1 2 3

1

2

3

4

5

6

7

8

9

10

Experiment number

Con

sum

er n

umbe

r

1 2 3

1

2

3

4

5

6

7

8

9

10

Experiment number

Con

sum

er n

umbe

r

Figure 6: Comparison of consumers selected for DR using three approaches during during time slots t = 13 (left) and t = 22(right). Note that at t = 13, the stochastic case results in certain non-intuitive choice of consumers (e.g. 2 and 4 which havesmall baselines, but higher participation probabilities.) Similarly, st t = 22, the stochastic case results in certain non-intuitivechoice of consumers (e.g. 2 and 3 which have small standard deviations)

1 2 3 4 5 6 7 8 9 100

0.5

1

1.5

2

2.5

3

3.5

Consumer number

Con

sum

ptio

n (k

Wh)

Baseline consumptionStandard deviationDR target (approach 1)DR target (approach 2)DR target (approach 3)

1 2 3 4 5 6 7 8 9 100

0.5

1

1.5

2

2.5

3

3.5

Consumer number

Con

sum

ptio

n (k

Wh)

Baseline consumptionStandard deviationDR target (approach 1)DR target (approach 2)DR target (approach 3)

Figure 7: Comparison of DR target consumptions for consumers during during time slots t = 13 (left) and t = 22 (right)using the first three optimization approaches. The baseline and standard deviation based on historical data are also shownfor reference. Note that at t = 13, while consumers with small baseline and standard deviation (e.g. 2 and 4) are nevertargeted in the deterministic case (approach 1), they can be targeted in the stochastic case (approach 2) due to their higherparticipation probabilities.

line consumption provide higher curtailment of demand thanconsumers with low baseline consumption. As a result ofthese observations, we expect that consumers with largestandard deviation and high baseline consumption shouldbe chosen for DR in approach 1. From Figure 6, it can beverified that the optimal choice of consumers for DR indeedconforms to the above intuition.The intuition stated above also holds for approach 2, af-

ter taking into account the probabilities of participation: weexpect that consumers with large standard deviation, highbaseline consumption and high probability of participationshould be targeted for DR. It is difficult to apply this in-tuition directly, since a set of nt consumers satisfying allthe above 3 criterion does not exist. Therefore, in Figure6, while we observe that some consumers satisfying thesecriterion (consumers 1 and 5) are selected, the selection ofremaining consumers (2 and 4) is non-trivial. This supportsthe use of a systematic optimization framework - such as theone proposed in this work - since intuition might be limitedand inadequate.Next, we compare the results of approach 1 and approach

2 to the results of the the rule based strategy (approach 3).

We first compare the expected reduction in aggregate con-

sumption(∆Qt

expected

)obtained using these approaches,

which is computed using the following equation:

∆Qt

expected = pi,t,dDR∆Qi,t,dDR (26)

The expected reduction in consumption in each of theseapproaches during timeslot t = 13 is 1.069 kW which ex-actly matches the desired reduction (

∑N1=1 Q

bi,t,dDR

−Qst,d).

The same observation was made for DR at timeslot t = 22.Therefore, we conclude that the consumpting mitigation ob-jective of DR is met in all these approaches. We also quan-tify the inconvenience caused to consumers in each of theseapproaches. This is done by computing the expected re-duction in utility from the maximum value of 1 for eachconsumer chosen to participate in DR, and adding it overall such consumers. The inconvenience caused to consumersfor each of these approaches and during each DR timeslot,as obtained using this methodology, is shown in Figure 8.We observe that for DR during timeslot t = 13, approach 3

13 220

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

Timeslot number for DR

Tot

al in

conv

enie

nce

caus

ed

Approach 1Approach 2Approach 3

13 220

5

10

15

20

25

Timeslot number for DR

Tot

al in

conv

enie

nce

caus

ed

Approach 1Approach 2Approach 3

Figure 8: Comparison of inconvenience caused (total reduction in utility over all consumers) using the first three approaches,when number of consumers is 10 (left) and when number of consumers is 500 (right). Note that for DR at timeslot t = 13,the inconvenience caused using heuristic rule-based DR strategy (approach 3) is almost twice that of iDR (approach 2).

results in almost twice as much inconvenience as that causedin approach 2. For timelot t = 22, the inconvenience causedis about 9% more from approach 2 than from approach 3.From these results, we conclude that the rule based strategyis at best suboptimal and reinforces the importance of usinga systematic optimization framework, such as the one pro-posed, over heuristic rule based strategies for DR planning.Note that we do not perform a comparison of approaches 1and 3 because they represent two different setups, the lat-ter corresponds to a stochastic setting whereas the former isdeterministic.Lastly, we re-run the afore-mentioned experiments for the

larger set of N = 500 consumers. The probabilities pi,t,dDR

were assigned arbitrarily, and we assume that the timeslotscorresponding to DR are still t = 13 and t = 22, whenthe utility’s desired supply power Qs

t,dDRfalls short of the

expected demand∑N

i=1 Qbi,t,DR by 10%. The DR design

variables ηtmax and ∆Qi,t,dDR and nt are set to 0.25 and 167

respectively. The observations made were similar to thosereported above for 10 consumers. The targeted DR reduc-tion was achieved in all these approaches and the inconve-nience comparison is presented in Figure 8.

5.7 Fairness assessmentThe objective of this experiment is to evaluate the fair-

ness strategy proposed in (18). For simplicity, we create 10identical clones of consumer 1 used in the previous threeexperiments, resulting in a group of consumers with similarconsumption preferences. The parameters nt in (10) andNmax

DR,i in (18) are both set to 5, and the timeslot t = 13 ischosen for simulations.Figure 9 shows the number of times each consumer is tar-

geted, as DR iterations progress. It is observed that afterthe end of a sufficiently large number of iterations (in thiscase 10), all consumers have been chosen an equal number oftimes. This verifies that the proposed DR planning strategyis ‘fair’ across all consumers in the group.

6. DISCUSSIONS AND FUTURE WORKIn this paper, we presented iDR, a DR planning methodol-

ogy that helps electricity providers in designing effective de-mand response events. In summary, iDR helps answer threeimportant questions: (i) when to plan DR events, (ii) which

1 2 3 4 5 6 7 8 9 10

0

1

2

3

4

5

6

Consumer number

Num

ber

of D

R s

igna

ls s

ent

Iteration 1Iteration 5Iteration 10

Figure 9: Progression of consumer selection as a function ofDR iterations. Note that each consumer has been selectedequal number of times at the end of 10 iterations, ensuringfairness

consumers to target, and (iii) what signals to send. Theproposed approach involved the estimation of baseline con-sumptions using historical consumption data obtained fromsmart meters, and then the development of utility functionsusing smart meter data to gauge the inconvenience associ-ated with consumers shifting their demand from baselines.These utility functions were then used in the solution of achance constrained, mixed integer nonlinear program. Theapproach was tested in simulation using data from a publicdata source. Results indicated that the DR objectives wereachieved, and iDR resulted in lower inconvenience than aheuristic rule based approach.

Based on the work presented in this paper, we identifythe following avenues of future research involving iDR. AGaussian utility function was used in the simulation exper-iments reported in Section 5. As an alternative, empiricallyderived utility functions from the consumption dataset canalso be implemented. However, the potentially additionalcomputational overhead introduced would need to be bal-anced against the value added by the use of such an empiri-cal, purely data driven approach. Secondly, the relationshipbetween DR participation probabilities and incentives needsdetailed investigation. This can enable an augmentation of

the existing capabilities of iDR. For example, it can enablethe development of appropriate incentives in order to alterthe participation probabilities of a set of consumers, suchas those with poor DR participation history. In general,the use of participation probabilities in the framework pro-vides a handle for the electricity providers to achieve otherDR objectives besides demand mitigation, such as the de-velopment of effective DR contracts. Thirdly, the mitigationof unwanted outcomes such as rebound effects [10] can beincluded in the optimization framework to increase its prac-tical value. Fourthly, extensions of the proposed approachto plan DR for consumer groups instead of individual con-sumers can result in potential advantages such as a reductionin computational overhead. Previous work in the area ofconsumer segmentation such as [19] can be used in this con-text. The development of appropriate optimization frame-works to aid DR planning in situations where appliance levelconsumption information is available is also planned for fu-ture. These frameworks would be based on the preferencemining methods presented in SEction 3.2. Lastly, we iden-tify the real-world testing of iDR through pilot studies togauge its true potential as a DR planning tool as an impor-tant final step in its development.

7. REFERENCES[1] N. Armaroli and V. Balzani, “The future of energy

supply: challenges and opportunities,”AngewandteChemie International Edition, vol. 46, no. 1-2, pp.52–66, 2007.

[2] U.S. Department of Energy, “Benefits of demandresponse in electricity markets and recommendationsfor achieving them,”urlhttp://eetd.lbl.gov/ea/emp/reports/congress-1252d.pdf,2006.

[3] K. Spees and L. Lave, “Impacts of responsive load inpjm: Load shifting and real time pricing,”The EnergyJournal, vol. 29, no. 2, pp. 101ı£¡–122, 2008.

[4] Federal Energy Regulatory commission, “A NationalAssessment of Demand Response Potential,”http://www.ferc.gov/legal/staff-reports/06-09-demand-response.pdf,2009.

[5] A. Chardon, O. Almen, P. E. Lewis, J. Stromback,and B. Chateau, “Demand Response: a decisivebreakthrough for Europe. How Europe could saveGigawatts, Billions of Euros and Millions of tons ofCO2.” Capgemini, Tech. Rep., June 2008.

[6] Smart metering project in the United Kingdom,“Smart Metering Projects Map,” http://bit.ly/GSs1o5,2013.

[7] Peak Load Management Alliance (PLMA), “DemandResponse: Design Principles for Creating Customerand Market Value,” http://www.peaklma.com,November 2002.

[8] US Department of Energy, “Benefits of demandresponse in electricity markets and recommendationsfor achieving them,” Feb. 2006.

[9] Fred Yoo, Christine Riker, “PG&E AutomatedDemand Response Program: IDSM Lessons Learned.”

[10] D. P. Seetharam, A. Agrawal, T. Ganu, J. Hazra,V. Rajaraman, and R. Kunnath, “Hidden costs ofpower cuts and battery backups,” in Proceedings of the

fourth international conference on Future energysystems. ACM, 2013, pp. 39–50.

[11] T. Bapat, N. Sengupta, S. K. Ghai, V. Arya, Y. B.Shrinivasan, and D. Seetharam, “User-sensitivescheduling of home appliances,” in Proceedings of the2nd ACM SIGCOMM workshop on Green networking.ACM, 2011, pp. 43–48.

[12] A. Agnetis, G. de Pascale, P. Detti, and A. Vicino,“Load scheduling for household energy consumptionoptimization.”

[13] T. X. Nghiem, M. Behl, R. Mangharam, and G. J.Pappas, “Scalable scheduling of building controlsystems for peak demand reduction,” in AmericanControl Conference (ACC), 2012. IEEE, 2012, pp.3050–3055.

[14] J. Zhu, L. Shen, R. Yin, and Y. Lu, “A two-stagesimulation-based on-line optimization scheme for hvacdemand response,” 2012.

[15] Z. Baharlouei, M. Hashemi, H. Narimani, andH. Mohsenian-Rad, “Achieving optimality and fairnessin autonomous demand response: Benchmarks andbilling mechanisms,” 2013.

[16] N. Li, L. Chen, and S. H. Low, “Optimal demandresponse based on utility maximization in powernetworks,” in Power and Energy Society GeneralMeeting, 2011 IEEE. IEEE, 2011, pp. 1–8.

[17] M. Kanakakis, M. Minou, C. Courcoubetis, G. D.Stamoulis, and G. Thanos, “A practical iterativeprice-based approach for optimal demand-response.”

[18] A. Albert, R. Rajagopal, and R. Sevlian, “Segmentingconsumers using smart meter data,” in Proceedings ofthe Third ACM Workshop on Embedded SensingSystems for Energy-Efficiency in Buildings. ACM,2011, pp. 49–50.

[19] T. K. Wijaya, T. Ganu, D. Chakraborty, K. Aberery,and D. P. Seetharam, “Consumer Segmentation andKnowledge Extraction from Smart Meter and SurveyData,” in SDM, 2014.

[20] K. Inc., “Pjm empirical analysis of demand responsebaseline methods,” 2011.

[21] EnerNOC, “The demand response baseline,” 2011.

[22] W. Charytoniuk, M. Chen, and P. Van Olinda,“Nonparametric regression based short-term loadforecasting,” Power Systems, IEEE Transactions on,vol. 13, no. 3, pp. 725–730, 1998.

[23] W. Shen, V. Babushkin, Z. Aung, and W. L. Woon,“An ensemble model for day-ahead electricity demandtime series forecasting,” in Proceedings of the fourthinternational conference on Future energy systems.ACM, 2013, pp. 51–62.

[24] “Electricity customer behaviour trial,” TheCommission for Energy Regulation (CER), 2012.

[25] “Enernoc: Open data to facilitate research anddevelopment,” http://open.enernoc.com/data/, 2013.

[26] S. Le Digabel, “Algorithm 909: Nomad: Nonlinearoptimization with the mads algorithm,”ACMTransactions on Mathematical Software (TOMS),vol. 37, no. 4, p. 44, 2011.

[27] Opti Toolbox - A free MATLAB toolbox foroptimization,www.i2c2.aut.ac.nz/Wiki/OPTI/.