1 Electric Vehicles in smart grids: a Hybrid Benders/EPSO Solver for Stochastic Reservoir Optimization Vladimiro Miranda Fellow IEEE Administration Board INESC TEC President INESC P&D Brasil Hrvoje Keko vladimiro.miranda@inesctec.pt hrvoje.keko@inesctec.pt
EV-based Stochastic Storage Concept - a group of vehicles connected to the grid comprises a stochastic storage - all variables are time dependent and stochastic (incl. max and min energy storable in the reservoir) - partial contributions from each electric vehicles as seen from the generation system form a cluster of vehicles = stochastic reservoir 2
EV-based Stochastic Storage Concept energy Emax Ed : energy desired by this cluster Ea(t=t 0 ) { Def amount provided Ea(t=t end ) The desired energy Ed: the difference between the final state of charge of an aggregated reservoir and the initial state start Emin end time The energy deficit is the difference between desired and actual SoC at the time of disconnection; analogous to ENS 3
Time Decomposition of Charging: Network Flow Model E r (c=1, t=0) P(t=0) P(t=1) P(t=2) cluster 1 E r (c=1, t=2) cluster 2 P(t=3) E r (c=2, t=3) The constraints are formulated as flow constraints, decomposed per time period. Energy that the power system can supply to / draw from the reservoir is limited: flow limits (aggregated interface limitations, grid limits etc). EE rr tt, cc EE rr tt, cc EE rr tt, cc, c = vehicle cluster cc, tt After a group of vehicles is disconnected, the power cannot flow from the system to it (and vice versa) so there s no connection (e.g. cluster 1 in t=3) 4
Transition to V2G: three EV Charging Modes - dumb charging: - time of disconnection is equal to the end of charging, no hourly arbitrage possible, inflexible demand (not flexible regarding power drawn from the grid) tt eeeeee cccccccccccccc = tt ccccccccccccccc EE rr tt, cc = EE rr tt, cc, tt - unlike additional load: the cost of failing to provide the desired amount of energy may be different from ENS! - smart charging: - permits hourly arbitrage delayed charging - charging amount controllable - does not permit the energy flow from the vehicle to the grid tt=tt 1 EE rr tt, cc EE dd cc EE rr (tt, cc) tt=tt 0 cc EE rr tt, cc 0, tt EE rr tt, cc EE rr tt, cc, tt cccccccc, tt 0 cc tt cccccccc tt ff (cc) 5
Transition to V2G: Three EV Charging Modes - vehicle to grid interface charging: - the real stochastic reservoir: EV pool is a stochastic storage - energy can flow in both directions - the equations link the time periods EE rr tt, cc EE dd cc EE rr (tt, cc) EE rr tt, cc EE rr tt, cc EE rr tt, cc tt=tt 1 tt=tt 0 cc tt=tt 1 EE rr tt, cc < 0 EE dd cc EE rr tt, cc EE mmmmmm tt, cc tt=tt 0 cc, tt cccccccc, tt 0 cc tt cccccccc tt ff (cc), tt - the model permits modeling gradual implementation of V2G connections 6
The Energy Deficit (in EV reservoirs) - The unit commitment cost function becomes: Minimize CCCCCCCC PP tt + tt CCCCCCCC DDDDDD cc cc subject to system restrictions - i.e. minimize the system operational cost plus the energy deficit cost : the price of not completely providing adequate charging to the EV owners - the inclusion of the deficit makes the problem always mathematically feasible, favorable for cut generation and algorithm speed (only optimality cuts are generated) - different from the cost of energy not being supplied (ENS) - aggregators may have a diversity of contractual obligations with the EV owners - this is the cost of customer dissatisfaction 7
STRUCTURE OF THE PROBLEM Benders and Dual Dynamic Programming: basics Min Subj: t t 1 1 t n C X + D Y +... + D Y AX n n n n n b J X + K Y g 1 1 1 1......... J X + K Y g Min CX t t t 1 1 + CX 2 2 +... + CnXn A1X1 b1 EX 1 1 + A2X2 b2 Subj : E2X2......... + AnXn bn a set of cascading independent problems: Min CX t t t 1 1 + CX 2 2 +... + CnXn A1X1 b1 + A2X2 b2 EX 1 1 Subj :...... + AnXn bn En 1Xn 1 8
BENDERS master and slaves MASTER (integer) Slave 1 Slave (LP) Slave 2 (LP) Slave n (LP) 9
Stochastic extension Admit a problem with two time stages and with two scenarios for the second stage. 21 22 (scenarios) 1 2 The structure of the problem may be Time stages Min C t t t 1 X1 + p1c2x21 + p2c2x22 A1X1 b1 Subj : E2X1 + A2X21 b21 E2X1 + A2X22 b22 where p 1, p 2 are the probabilities of each scenario
The Benders trick Primal and dual forms of the sub-problem: t ( X ) min α n 1 = CnXn Subj : AnXn b En 1Xn 1 Xn 0 ( b E X ) t α= Max n 1 n 1 π Subj.: A t nπ Cn π 0 The domain of the dual does not depend on X n-1!!! The optimum of the dual may be found among the vertices of a fixed domain (if X n-1 changes, the optimum may change vertex, but the feasible region is constant) 11
Solving the dual Searching for vertices Two different slopes for the objective function, from different values of X α = Subj.: Max ( g JX) K t π π 0 t π D min α α α... α ( g JX) ( g JX) ii t n ( g JX) π t t π π i For a given X*, each vertex defined by the dual variables π will have an α value - and we wish to select the vertex π* with maximum α Benders
Adding a constraint to the master Solving the sub-problem for a given X*, we find a vertex of the dual, which corresponds to a valid constraint that can be added to the master: Min Subj: ( g JX) If we solve now the master, we get a new value for X* which will allow finding another vertex π in the sub-problem dual which will allow a new constraint to be added to the master which... α t C X + α AX b t π * BUT Adding a constraint to the problem may be replaced by adding a penalty to the Master objective function THE MASTER PROBLEM MAY BE SOLVED BY A META-HEURISTIC WITH EVOLVING LANDSCAPE! Benders
Stochastic modeling: scenarios - In systems with storage, the decisions in a time step are reflected on other time steps - snapshot analysis - considering each time period as independent from others is not adequate - sequence of marginal distributions also inappropriate! - missing temporal evolution of variables - The chosen model of uncertainty: scenarios - Sampled from an estimator model - for wind power: covariance matrix estimation - for EV behavior: Gaussian copula-based Monte Carlo model or extracting data from agent-based model simulations of traffic behavior - Result: a large set of sampled scenarios -> requires clustering to reduce the number of scenarios! 14
Sequential Monte Carlo generating patterns Representing distinct behavior models: Methodic citizens (charging the EV at the end of the day only) Obsessed citizens (charging the EV whenever possible) Relaxed citizens (charging the EV only when the battery is empty) 120 100 y ( ) 80 60 40 20 0 Monday Tuesday Wednesday Thursday Friday Saturday Sunday SOC (%) of the EV battery 15
Stochastic modeling: scenario reduction - From a large set of Monte Carlo sampled scenarios, a clustering process delivers a set of weighted scenarios according to a similarity metric - Clustering problem: maximize entropy among clusters, minimize entropy within each cluster, assign relative weights - In the cases tested, the distance metric used was the absolute per-hour deviation 16
Unit Commitment with Renewables and Stochastic Storages The day-ahead stochastic UC problem decomposed into three stages: Stage 0 Master problem Stage 1 Stage 2 binary decisions min up and down time constraints startup and shutdown continuous variables ramping constraints decomposed hydro constraints (max energy per day) max inclusion of renewable power min customer dissatisfaction 17
(Classic) UC Problem Formulation - Quadratic fuel cost functions of thermal units, piecewise linearized per interval CC gg (tt) = aa uucc gg tt + bbpp gg tt + ccpp gg tt 2 - Start-up and shut down costs SSUU gg tt = uucc gg tt uucc gg tt 11 SSUU gg - Hydro generators with large storage: total energy produced during the day is constrained (from long term optimization governing classic storage operation) EE gg,mmmmmm PP gg tt EE gg,mmmmmm tt - Min up time and min down time constraints + unit initial conditions: s=1 if unit changes state, 0 otherwise tt tt gg,oooo ii tt ss gg,oooo ii uucc gg (tt), ss gg,oooooo ii 11 uuuu gg (tt) tt tt gg,ooffff ii tt - Generation limits - Ramping limits PP gg,mmmmmm PP gg tt PP gg,mmmmmm RR gg,dddddddd PP gg tt PP gg tt 11 RR gg,uuuu 18
EPSO and the generalized version DEEPSO The Master problem on integer variables) is solved by a customized version of EPSO algorithm DEEPSO concept: o inertia: moving in the same direction inertia (memory) b i X new o perception: sensing a local gradient (by the swarm) X perception X r1 o cooperation: attraction to the proximity of the global best new new X = X+ V new * * * * V = wi V+ w M( Xr1 X) + w C P ( bg X) X old cooperation * subject to mutation P communication probability b G * b G
EVOLVING SWARMS EPSO AND DEEPSO EPSO the gradient perception is based on the particle self-memory term V new = w * * * * I V+ w M( bi X) + w C P ( bg X) DEEPSO a flavor of Differential Evolution added to EPSO new * * * * V = wi V+ w M( Xr1 X) + w C P ( bg X) Variants : sampled among the current generation : Sg sampled among the matrix b i of individual past bests : Pb X r1 as a uniform recombination of the current generation : Sg-rnd as a uniform recombination within the matrix b i : Pb-rnd for the latter 2: not taking in account the direction of ( Xr1 X) :. - minus taking in acc. the direction of ( Xr1 X) :. + plus taking in acc. the direction of ( Xr1 X) in each coordinate:. 0 zero DEEPSO: winner of the 2014 IEEE competition of m-h for the OPF problem 20
Custom EPSO for UC (Stage 0) - Initialization of unit commitment status: - Custom tailored heuristic, not general but with more insight into the problem - Heuristic order-based rule to commit units and construct initial population of solutions - Commit enough units until sum(p) is enough to cover the max load - Checks and repairs for violating of on/off restrictions (commiting and decommiting when violated) - Each particle maps the {0,1} space of unit commitment decisions to fitness value EACH OF THE SCENARIOS IN STAGES 1 and 2 ESSENTIALY REPRESENTS AN ADDITIONAL PENALTY TO THE FITNESS LANDSCAPE! 21
Linearized subproblems in stages 1 and 2 - The subproblems involving wind power and storage are formulated as LP problems - Solved using a industry standard LP solver to optimality the dual is easily obtained - The linear formulations in stage 1 include PNS and in stage 2 include Energy Deficit: - There always exists a mathematically feasible solution - Faster to solve to (mathematical) feasibility even if technical feasibility is not obtained - The cuts generated by the subproblems: they are optimality cuts and not feasibility, so no distortion in the EPSO search space 22
Scenarios as penalties to the fitness function: risk handling In the expected value formulation, the fitness function is CCCCCCCC = EE CCCCCCCC = 11 NN ss CCCCCCtt ss ss SS For a robust optimization (worst case), the worst case defines the cost CCCCCCCC = mmmmxx ss SS CCCCCCtt ss For a minimax regret formulation, the cost is the maximum cost deviation from what would be the cost with perfect foresight (if the particular scenario occurred exactly) CCCCCCCC = mmmmxx ss SS CCCCCCtt ss CC pppp (ss) 23
Scenarios as penalties to the fitness function An adaptive scheme handles the constraint contributions to the fitness function ff = λλ ss pp ss CCCCCCtt ss ss Weight factors for each scenario cost are adapted according to the risk model and the probability of a certain scenario Remember: the scenarios have a probability as a result of clustering Exponential decay is used to forget scenarios that do not discriminate between the current population of solutions λλ ss,ii+11 = μμ λλ ss,ii ; μμ < 11 In the initial phase of the algorithm a limited subset of scenarios is used to speed up convergence and then resampling is used 24
Finishing an iteration of the algorithm In the classic EPSO, the location information is shared between individuals; here, the solutions also share scenario values Optimal cost values for stage 2 and reduced cost coming from the equality constraint are shared! This way the stage 2 solutions can be approximately calculated using the Benders decomposition cost-to-go principle! Cut calculation includes historical values: no recalculation if not necessary After each iteration, binary solutions of stage 0 are repaired if there are violations in the startup and shutdown time constraints 25
Results: comparison with a MILP solver (GUROBI) - classic formulation with 1000 wind power scenarios and 5 units - can be solved both with MILP solver directly - the same solver used within EPSO for LP problems used here for the whole problem - stable and consistent results of EPSO with comparable performance - approx 20% loss in performance provides more flexibility Average run times: 5 unit 1000 wind scenarios (no reduction), 100 repeated runs, expected value formulation: MILP UC: 121 seconds EPSO UC: 163 seconds (on a i7 3820qm laptop computer) 26
Results: 10 Unit illustration problem (1) - 1000 wind power scenarios reduced to a set of 20 scenarios - 500 scenarios of EV integration reduced to a set of 20 scenarios - 10 unit thermal system, enough power but relatively constrained with regard to flexibility (ramps) - no electric vehicles, wind power integration only: 27
Results: 10 Unit illustration problem (2) - 33.000 vehicles with 16 kwh average desired energy, on avg 550 MWh of additional energy required throughout the day - 80% of vehicles dumb charging, 20% smart charge, no V2G 1.8% additional energy -> 3.1% increase in system cost! no PNS, wind not spilled, energy deficit at bay smart chargers help avoiding wind spill but the resulting additional load not favorable most covered by expensive generator 7 28
Results: 10 Unit illustration problem + strong V2G (3) - 20% of vehicles have smart charging and 80% V2G, no dumb charging the most expensive generator works far less additional load distributed more favorably: 1.8% additional load -> 2.0% increase in overall cost! - system cost reduced due to more charging in cheaper valley hours - some extra demand still occurs in peak hours; 2.0% increase in overall cost compared to NO EV case - 6.81% lower overall cost compared to dumb charging case - 35% less increase in system costs 29
CONCLUSIONS A Benders decomposition in the form of {Dual LP Slave problems + EPSO solver for the Master integer Problem} is successful! A stochastic model with high number of scenarios (wind + EV use) Allows the modeling of different strategies of EV integration Allows for several risk managing strategies Master problem optimized under an evolving landscape (Benders cuts transformed into penalties) Competitive performance on problems solvable with MILP allows the reasonable expectation of superior performance in high dimension problems, because of the Benders decomposition principle The algorithm is also promising for planning purposes (simulated pricing behavior) and for the TSOs and aggregators. 30