Large Scale Charging of Electric Vehicles: A Multi-Armed Bandit Approach

Transcription

1 Large Scale Chargng of Electrc Vehcles: A Mult-Armed Bandt Approach Zhe Yu, Yunan Xu, and Lang Tong Abstract The problem of centralzed schedulng of large scale chargng of electrc vehcles (EVs) by a servce provder s consdered. A Markov decson process model s ntroduced n whch EVs arrve randomly to the chargng faclty wth random demand and completon deadlnes. The servce provder faces random chargng costs, convex non-completon penaltes, and a peak power constrant that lmts the maxmum number of smultaneous actvaton of EV chargers. Formulated as a restless mult-armed bandt problem, the EV chargng problem s shown to be ndexable, thus low complexty ndex polces exst. A closed-form expresson of the Whttle s ndex s obtaned for the case when the chargng costs are constant. The Whttle s ndex polcy, however, s not optmal n general. An enhancement of the Whttle s ndex polcy based on spatal nterchange accordng to the less laxty and longer processng tme (LLLP) prncple s presented. The proposed polcy outperforms exstng chargng algorthms, especally when the chargng costs are dynamc. Index Terms Mult-armed bandt problem; deadlne schedulng; chargng of electrc vehcles; Whttle s ndex; Markov decson processes. I. INTRODUCTION ELECTRIC vehcles (EVs) and EV chargng servces have grown substantally. Snce 2011, the EV sale has grown 20 tmes, and EV chargng statons have ncreased 7.5 folds [1], [2]. Although most EVs n use currently are charged at prvate homes, there s a need to develop large chargng facltes wth fast chargng capabltes n publc spaces such as parkng garages, parkng lots at commercal locatons, and hghway rest stops. Such chargng facltes allevate range anxety of EV consumers; they are essental to the growth of EV market share [3]. EV chargng at facltes wth capacty of hundreds of EVs faces a dfferent set of techncal challenges from those assocated wth ndvdual home chargng. Frst, consumers expect chargng to be completed wthn a relatvely short perod of tme. Thus, fast chargng devces operated at hgh peak power becomes essental. Ths type of chargng, f unmanaged, may have detrmental effects on system relablty. It may be necessary to lmt the number of smultaneously actvated chargers. Second, there s sgnfcant uncertanty n chargng demand. EVs arrve at a chargng faclty randomly, each wth stochastc Z. Yu and L. Tong are wth the School of Electrcal and Computer Engneerng, Cornell Unversty, Ithaca, NY 14853, USA. Y. Xu s wth the School of Engneerng Systems and Desgn Pllar, Sngapore Unversty of Technology and Desgn, Sngapore, Emal: {zy73,lt35}@cornell.edu, yunan_xu@sutd.edu.sg. Ths work s supported n part by the Natonal Scence Foundaton under Grant CNS demand and random deadlnes, whch makes t dffcult for the scheduler to meet consumer demands. Thrd, the cost (or the proft) of the servce provder may be stochastc. For nstance, the servce provder may partcpate n the wholesale electrcty market and s subect to realtme prce fluctuatons. In addton, the servce provder may ntegrate local renewables such as solar generatons that are ntermttent. Fnally, the energy management system that schedules EV chargng needs to operate n real tme, thus must be scalable wth respect to the sze of the chargng faclty, whch rules out the use of brute-force optmzaton technques. A. Summary of results Ths paper presents an onlne schedulng algorthm that s computatonally scalable and capable of dealng wth demand and cost uncertanty. We ntroduce a constraned Markov decson process (MDP) model wth the obectve of maxmzng expected (dscounted) proft subect to a constrant on the maxmum number of smultaneously actvated chargers. The model captures randomness n arrvals, demand, deadlnes, and chargng costs. The optmal chargng problem s then reformulated as a restless Mult-Armed Bandt problem (MAB) wth smultaneous plays [4]. We establsh the ndexablty for the restless MAB problem. For the constant cost case, we obtan the Whttle s ndex n closed form, whch exposes certan weaknesses of Whttle s ndex polcy. Because Whttle s ndex polcy s not optmal n general, we present an mprovement of the Whttle s ndex polcy usng a spatal nterchange procedure based on the less laxty and longer processng (LLLP) tme prncple [5]. Numercal smulatons demonstrate mproved performance, especally when the cost of chargng s dynamc and traffc s relatvely heavy. B. Related work The centralzed EV chargng problem consdered n ths paper falls n the category of stochastc mult-processor deadlne schedulng problem. In that context, EVs are obs and chargers are processors. The work most relevant to the current paper s [6] by Raghunathan, Bokar, and Kumar on a deadlne schedulng problem n wreless communcatons. The authors of [6] are perhaps the frst to formulate the stochastc deadlne schedulng problem as a restless MAB problem and establshed ndexablty. Also related s [7] where the problem of schedulng packets wth deadlnes n ad hoc networks s consdered.

2 2 There are several nontrval dfferences between the models n [6], [7] and that n the current paper. For nstance, the arrval models used n [6] are ether smultaneous or perodc. The cost models n [6], [7] are also sgnfcantly dfferent from ours. The results n [6], [7] do not apply drectly here. The dynamc programmng approach to EV chargng was consdered n [5] where the Less Laxty and Longer Processng tme (LLLP) prncple was frst establshed. LLLP s an enhancement of any polcy va a spatal nterchange argument, and t s used n ths paper on the Whttle s ndex polcy. Earler, Panwar and Towsley consdered the case when there s a sngle processor [8], [9]. It was shown that the earlest deadlne frst (EDF) schedulng mnmzes the unfnshed work. See also the more recent work of Lehoczky and Shreve [10]. The performance of stochastc deadlne schedulng problem nvolvng multple processors s largely unknown. The deadlne schedulng problem was orgnally consdered by Lu and Layland [11] and studed extensvely under models wth determnstc arrvals, ob szes, and deadlnes. See [12], [13] for some of the classcal results. The sngle processor case s relatvely well understood. For the multprocessor problem, the results are lmted. In fact, optmal onlne schedulng does not exst n general [14]. Applcatons of determnstc deadlne schedulng to EV chargng are consdered n [15], [16], [17], [18]. Because arrvals and demands are arbtrary, performance measures used are often based on the worst case scenaros, whch may not be approprate for evaluatng the performance of EV chargng servces. Dfferent from the centralzed schedulng of EV chargng framework consdered here, there s a sgnfcant body of lterature on home EV chargng problems, often n decentralzed optmzaton or game theoretc settngs. For example, the work n [19] and [20] ams to mnmze the load varance. Under a game theoretc model, the authors showed that the Nash Equlbrum s the optmal valley fllng polcy. A decentralzed control algorthm for the same purpose s proposed n [21]. The vehcle to grd (V2G) ancllary servces are consdered n [22], [23], [24]. In [22], an optmal prcng strategy s proposed to encourage EVs to partcpate n frequency regulaton and a centralzed control s used n [24]. II. PROBLEM FORMULATION We now formulate the EV chargng problem as one of stochastc deadlne schedulng subect to a constrant on the number of servers (chargers). In Secton II-A, a constraned Markov Decson Processes (MDPs) problem s ntroduced. In Secton II-B, we provde an upper bound on the total dscounted reward, whch s useful for benchmark comparsons. A. An MDP formulaton of Stochastc Deadlne Schedulng Fg. 1 shows a schematc of an energy management system at an EV chargng faclty. We assume that the faclty has N parkng spots, each wth a charger that can be actvated or deactvated by the scheduler. The charger can only be Charger Chargng cost Scheduler Fg. 1: Archtecture of a chargng staton r t B [t] T [t] L [t] Fg. 2: An llustraton for the charger s state. r s the arrval tme of an EV at charger, d the deadlne for completon, B [t] the amount of chargng to be completed by d, T [t] the lead tme to deadlne. connected to one vehcle. It s open f t s not attached to a vehcle and closed or occuped otherwse. EVs arrve at chargers ndependently. At the arrval of charger, f the charger s open, the EV s attached to t, and the scheduler records the arrval tme r, as shown n Fg. 2. The EV owner communcates the chargng demand B, measured n chargng tme, and deadlne for completon d to the scheduler, also shown n Fg. 2. We summarze the assumptons n the paper; they are approxmatons of practcal operatng condtons and are made for tractable analytcal developments. A1. Each charger can be connected to only one EV, and t s removed from the EV at the deadlne d. An EV s charged at a fxed rate normalzed to 1. A2. The EV arrvals to the N chargers are ndependent and dentcally dstrbuted (..d.). A3. The prce of chargng s proporton to the chargng demand, normalzed to 1 dollar/hour. A4. The margnal chargng cost c[t] s an exogenous fnte state Markov chan whose evoluton s ndependent of the state evoluton and actons of chargng. A5. The chargng of EVs s preemptve wthout cost. A6. The penalty for ncomplete chargng s a convex functon of the ncomplete amount at the deadlne. d tme

3 3 We now present elements of the dscrete-tme MDP n whch tme, ndexed by t = 0,1,2,, s slotted. At the begnnng of the slot, the system state s revealed to the scheduler and a decson on whch chargers to actvate or deactvate n the current slot s made and executed. 1) State space: The state of the chargng system S[t] = (c[t],s 1 [t],,s N [t]) S c S 1 S N s defned by the chargng cost c[t] and states of ndvdual chargers S [t] where S c s the state space of the cost and S the state space of ndvdual chargers. Specfcally, the state of chargers defned bys [t] =(T [t],b [t]) where, as llustrated n Fg. 2,T [t] d t s the lead tme andb [t] the remanng chargng demand measured n chargng tme. If there s no EV attached to charger, then S [t] = (0,0). The system chargng cost c[t] s the cost of electrcty from the wholesale market, offset by possbly locally generated renewables. 2) Acton: The acton of the scheduler s defned by a[t] = (a 1 [t],,a N [t]) {0,1} N where a [t] = 1 means that the charger s actvated (actve) whereas a [t] = 0 means that the charger s deactvated (passve). 3) State evoluton: We assume that the chargng cost c[t] (S c,p) s an exogenous fnte state Markov chan, ndependent of the actons of the scheduler and ndvdual state evolutons, wth transton probablty matrx P = [P, ]. Gven the scheduled acton a[t] = (a [t]), the evoluton of states at ndvdual chargers are assumed statstcally ndependent. When the charger s actve and the vehcle has postve remanng demand, both the chargng demand and the lead tme are reduced by 1. If the chargng demand of an EV s fulflled (B [t] = 0), then only the lead tme s decreased by one. EVs leave at ther deadlnes and new EVs arrve followng a geometrc dstrbuton and the state probablty mass functon (PMF) Q(, ). Specfcally, the state of charger wth state S [t] under acton a [t] = 1 s transtoned to ( ) S [t+1] a [t] = 1 (T [t] 1,B [t] 1) fb [t] > 0,T [t] > 1, (T [t] 1,B [t]) fb [t] = 0,T [t] > 1, (0,0) w.p. (1 ρ), f T [t] 1, (1,1) w.p. ρq(1,1), f T [t] 1, (T max,b max) w.p. ρq(t max,b max), ft [t] 1, (1) = where ρ s the probablty that an EV arrves at an empty charger and Q(, ) the PMF of the state of a newly arrved EV. Smlarly, the state of charger under the passve acton a [t] = 0 has the state transton gven by ( ) S (t+1) a [t] = 0 = (T [t] 1,B [t]) (2) 4) Reward: At tme t, the reward receved from charger wth acton a s gven by R a(s [t],c[t]) = (1 c[t])a, f B [t] > 0, T [t] > 1, (1 c[t])a F(B [t] a), f B [t] > 0, T [t] = 1, 0 otherwse, (3) wheref(b) s an ncreasng and convex penalty functon wth F(0) = 0. Note that the above reward functon means that the EV owner wth chargng demand B s charged B dollars f the chargng request s fulflled at the deadlne andb B F(B ) f there s B unfulflled chargng. Here F(B ) s the extra compensaton for unfulflled chargng. Gven the ntal system state S[0] = s and a polcy π that determnes a sequence of actons a[t],t = 0,1,, the expected dscounted system reward s defned by V π (s) =E π ( t=0 =1 ) N β t R a[t](s [t],c[t]) S[0] = s where E π s the condtonal expectaton for gven schedulng polcy π and 0 < β < 1 the dscount factor. 5) Constraned MDP and optmal polcy: We mpose a constrant on the number of smultaneously actvated chargers, N a [t] M for all t. In practce, such a constrant lmts the peak power consumpton by the servce provder, whch s necessary to satsfy the feeder and lne lmts. The optmal EV chargng polcy can now be formulated as a constraned MDP. Specfcally, let V(s) = {π: N (4) sup V π (s) (5) aπ [t] M t} where a π [t] s the acton generated by π. A polcy π s optmal f V π (s) = V(s). Wthout loss of generalty, we restrct ourselves to the set of statonary polces [25]. B. A performance upper bound In (5), the power lmt must be satsfed for all t. By relaxng ths constrant and requrng that the average power usage does not exceed M, we obtan a performance upper bound for (5). In partcular, a relaxed problem can be stated as { } N sup π E π t=0 =1 βt R a[t](s [t],c[t]) S[0] subect to (1 β)e N t=0 =1 βt a [t] M. (6) Problem (6) s not a practcal formulaton of the large scale EV chargng problem snce the power usage could be far more than M at some tme. Snce the chargng cost s the same for all chargers, the relaxed problem (6) s equvalent to the followng problem (on the schedulng of a sngle charger ). { sup π NE π t=0 βt R a[t](s [t],c[t]) S [0],c[t] } subec to (1 β)e t=0 βt a [t] M/N. (7) Problem (7) s to maxmze the dscounted reward from a sngle charger wth no more than M/N actve acton (per

4 4 tme perod) on average. The optmal soluton and the optmal obectve of (7) are the same as those of (6). The optmal obectve of (7) can be used as a performance upper bound for the orgnal schedulng problem n (5). The constraned MDP problem n (7) has a much smaller dmenson and can be easly solved by lnear programmng (cf. Chap. 3 of [25] for a survey). III. INDEX POLICY AND WHITTLE S INDEX The MDP formulaton does not result n a scalable schedulng polcy. Because chargng at ndvdual chargers s ndependent condtoned on chargng cost, we seek to obtan an ndex polcy that provdes a scalable soluton. By ndex polcy we mean that the schedulng s based on the ranked order of ndces assocated wth chargers. Specfcally, the ndex of charger s a mappng from ts extended state S [t] (S [t],c[t]) to a ndex value. A. Deadlne schedulng as a restless MAB problem We now formulate Problem (5) as a restless Mult-Armed Bandt (MAB) problem. The restlessness s due to the fact that the lead tme of each charger evolves even f the charger s not actvated. A complcaton of castng (5) as a restless MAB problem comes from the nequalty constrant on the maxmum number of smultaneous actvatons. Ths complcaton can be crcumvented by ntroducng M dummy chargers to the schedulng problem. Specfcally, each dummy charger accrues zero reward regardless of the actons appled to t, and the state of dummy chargers stays at S = (0,0). Wthout loss of generalty, let {1,,N} be the regular chargers and {N +1,,N +M} be the dummy chargers. 1) Arms: We let each charger be an arm. Defne the extended state of each charger as S [t] (S [t],c[t]) and denote the extended state space as S S S c. The actons and the reward functons reman unchanged. For the regular chargers, snce the cost dynamc s ndependent of the state and actons of chargng, the state transton of arm can be easly wrtten accordng to charger transton (1), (2) and cost transton P. For the dummy chargers, only the chargng cost evolves accordng to P. 2) MAB formulaton: By ncludng dummy chargers, the MDP n (5) can be vewed as a restless MAB problem that, at each tme, exactly M out of N + M chargers (arms) are actve. The optmzaton problem wth the equalty constrant s state as followng: { sup π E N+M π t=0 =1 β t R a[t]( S [t]) S } [0] (8) N+M s.t. =1 a [t] = M, t. It can be shown that the optmzaton wth the nequalty constrant defned Problem (5) s equvalent to the MAB nvolvng dummy chargers wth the equalty constrant n (8). B. Whttle s ndex We now examne the Whttle s ndex polcy for the restless MAB problem defned n (8). To ths end, we frst ntroduce Whttle s ndex and establsh n Sec III-C the ndexablty of the restless MAB problem n Theorem 1. Consder the followng sngle arm reward maxmzng problem wthout constrant: gven the ntal state S [0], } V ( s) sup π { E π β t R a[t]( S [t]) S [0] = s t=0, (9) wherev s the value functon. LetL a be the Markov transton operator on the extended state S and an arbtrary functon f( S ) defned as (L a f)( s) E{f( S [t+1]) S [t] = s,a [t] = a}. The maxmum dscounted reward of Problem (9) s determned by the Bellman equaton V ( s) = max{r 0 ( s)+β(l 0 V )( s),r 1 ( s)+β(l 1 V )( s)}. The Whttle s ndex s defned by ntroducng a subsdy ν pad to the scheduler to take the passve acton [4]. The Bellman equaton for the ν-subsdy problem s gven by V ν ( s) = max{r 0( s)+ν+β(l 0 V ν )( s),r 1( s)+β(l 1 V ν )( s)}, where V ν s the value functon for the ν-subsdy problem. Intutvely, the larger the subsdy ν s, the more lkely the passve acton would be optmal. Let S (ν) denote the set of arm states n whch t s optmal to take the passve acton on arm n the ν-subsdy problem. The ndexablty of an MAB problem s defned as follows. Defnton 1 (Indexablty): Charger (arm) s ndexable f the set S (ν) ncreases monotoncally from to S as ν ncreases from to +. The MAB problem s ndexable f all the chargers (arms) are ndexable. Gven the defnton of ndexablty, the Whttle s ndex s defned as follows. Defnton 2 (Whttle s Index): If charger (arm) s ndexable, ts Whttle s ndex ν ( s) of the extended state s s the nfmum subsdy ν such that the passve acton s optmal at state s,.e., ν ( s) nf ν {ν : R 0( s)+ν+β(l 0 V ν )( s) R 1 ( s)+β(l 1 V ν )( s)}. C. Indexablty and ndex closed-form In ths subsecton, we apply the Whttle s ndex to chargers (arms) n problem (8). In the followng theorem, we establsh the ndexablty of each charger; for the case wth constant chargng cost, we also derve the closed-form expresson of Whttle s ndex. Theorem 1 (Indexablty and ndex closed-form): 1) Each charger as an arm s ndexable. 2) If c[t] = c 0 for all t, Whttle s ndex of a regular charger {1,,N} s gven by

5 5 0 f B = 0, 1 c ν (T,B,c 0 ) = 0 f 1 B T 1, 1 c 0 +β T 1 [F(B T +1) F(B T)] f T B. (10) Whttle s ndex of a dummy charger s zero. ν (0,0,c 0 ) = 0, {N +1,,N +M}. In (10), when t s feasble to fulfll the EV s chargng request, ts Whttle s ndex s smply the chargng proft 1 c 0. When the penalty s nevtable, the ndex takes nto account both the chargng proft and the non-completon penalty. We note that the Whttle s ndex gves hgh prorty to urgent EVs wth non-postve laxty. Here, the laxty of chargers defned as L [t] T [t] B [t] (cf. Fg. 2). We note, however, that the Whttle s ndex does not dstngush EVs wth postve laxty. In the next secton we wll ntroduce an enhanced heurstc polcy based on the Whttle s ndex. The proof of Theorem 1 s omtted due to the space lmt. A proof sketch s provded n the followng. 1) Dummy Chargers: For {N +1,,N +M}, there s no EV arrval, and only the chargng cost evolves. The Bellman equaton of the ν-subsdy problem s gven by V ν (0,0,c ) = max{β k P(,k)V ν (0,0,c k), ν +β k P(,k)V ν (0,0,c k)}. When ν < 0, t s optmal to actvate the dummy charger. Otherwse, passve acton s optmal. So a dummy charger s ndexable and ts Whttle s ndex s ν (0,0,c ) = 0, accordng to Defnton 2. 2) Regular Chargers: For {1,,N}, the ndexablty can be proved by nducton. We frst fnd out Whttle s ndex for a charger wth T 1. Assumng Whttle s ndex exsts for T, we show that for T +1, the dfference between actve and passve actons s monotoncally ncreasng n ν and has a zero pont. Thus Whttle s ndex exsts for T +1. IV. WHITTLE S INDEX POLICY WITH LLLP INTERCHANGE For the obectve of tme average (β = 1) proft maxmzaton, Whttle s ndex polcy s shown to be asymptotcally optmal, as the number of arms ncreases to nfnty [26], under some condtons. For the dscounted proft maxmzaton settng consdered n ths paper, the asymptotc optmalty of Whttle s ndex polcy s not clear. For small systems wth fntely many arms, there are counter-examples where an optmal ndex polcy does not exst (and therefore the Whttle s ndex polcy cannot be optmal). In ths secton, we wll apply the Less Laxty and Longer Processng tme (LLLP) prncple (proposed n [5]) to mprove the Whttle s ndex polcy. A. Less Laxty and Longer Processng tme prncple The LLLP prncple s a prorty rule for the schedulng of chargng multple EVs, whch s defned as follows. Defnton 3 (LLLP Prorty): Consder chargers (arms) and at tme t. We say domnates ( ), f has Less π π B [t] actve passve actve passve T [t] T B [t] [t] t... Fg. 3: The LLLP nterchange τ actve passve actve passve tme Laxty and Longer Processng tme,.e., B [t] B [t] and L [t] L [t], wth at least one of the nequaltes beng strct. LLLP defnes a partal order over the EVs states such that the EV wth less laxty and longer remanng chargng demand should be gven prorty. In [5], the authors appled nterchange argument to show that LLLP could mprove the performance of any gven polcy along every sample path, and further, there exsts an optmal statonary polcy that follows the LLLP prncple under mld condtons. The LLLP nterchange can be easly mplemented to mprove any gven polcy π. As llustrated n Fg. 3, suppose that at tme t, EV has less laxty and longer remanng chargng demand than EV ( ), and that the polcy π charges but not. An LLLP nterchange mproved polcy π charges but not at tme t. Let τ [t + 1,mn{d,d }] denote the tme perod at whch π charges but not for the frst tme; at tme τ, π charges but not. If such a perod τ does not exst, then the nterchangng polcy π wll take the same acton as the orgnal polcy π after tme t. The LLLP prncple results n many smaller unfnshed obs rather than few large unfnshed obs and thus mproves the proft performance under tme-varyng chargng costs and convex non-completon penalty functon. The LLLP prncple wll be used to mprove the Whttle s ndex polcy. B. Index Polcy wth LLLP nterchange In ths subsecton, we propose a heurstc polcy: the Whttle s ndex polcy wth LLLP nterchange. The heurstc polcy can be obtaned by mplementng Algorthm 1. The proposed polcy takes the advantage of tme varyng chargng cost whle balancng the rsk of non-completon penaltes. In prncple, t gves hgher prorty to EVs wth

6 6 tght deadlnes and large remanng demand to avod potental penaltes. Algorthm 1 Whhttle Index wth LLLP nterchange 1. Calculate the Whttle s ndex of all chargers and sort them n a descend order. 2. Apply LLLP nter-change to the sorted chargers. 3. Actvate the M chargers wth hghest prorty. Total reward x 104 Dynamc prce: β= EDF LLF Whttle s ndex Index w. LLLP nterchange Upper bound 1.5 V. NUMERICAL RESULTS In ths secton, numercal experments are conducted to compare the performance of proposed ndex polcy wth other smple heurstc (ndex) polces,.e., EDF (earlest deadlne frst), LLF (least laxty frst) and Whttle s ndex wthout LLLP nterchange. If feasble, EDF charges M EVs wth the earlest deadlnes, and LLF charges M EVs wth the least laxty. Both polces wll fully utlze the capacty and charge M EVs as long as there are at least M unfnshed EVs n the system. Whttle s ndex polcy, on the other hand, ranks all chargers by the Whttle s ndex and actvates the frst M, and may put some (regular) chargers dle when the cost s hgh. We frst consder a specal case of Problem (8) wth a constant chargng cost. Snce the chargng cost s tme-nvarant, t s optmal to fully utlze the chargng capacty M to charge unfnshed EVs. We observe from Fg. 4 that the Whttle s ndex polcy wth LLLP nterchange and LLF acheve smlar performance, snce both polces roughly follow the least laxty frst prncple. The performance of these two polces s close to the performance upper bound. The EDF polcy performs poorly because t does not take the remanng chargng demand nto account. The gap between the Whttle s ndex polcy and the ndex polcy wth LLLP nterchange comes from the reorderng of EVs wth postve laxty (cf. the dscusson followng Theorem 1). Total reward 3.5 x 104 Constant cost: β= EDF LLF Upper bound Whttle s ndex Index w. LLLP nterchange # of regular chargers Fg. 4: Performance comparson: constant chargng cost c[t] = 0.5, ρ = 0.7, T max = 12, B max = 9, β = 0.999, M/N = # of regular chargers Fg. 5: Performance comparson: dynamc chargng cost, ρ = 0.7, T max = 12, B max = 9, β = 0.999, M/N = 0.5. For the dynamc chargng cost case, we use the realtme prcng sgnal from NY Independent System Operator (NYISO) and tran a Markovan model that descrbes margnal chargng costs. The Markov chan of chargng cost has a perod of 24 hours (a sngle day). In Fg. 5, the performance of dfferent polces s compared. Both EDF and LLF seeks to actvate as many regular chargers as possble, up to the capacty constrant M. The Whttle s ndex polcy, on the other hand, takes the advantage of the prcng fluctuaton and charges more EVs at prce valley and keeps some chargers dle when the cost s too hgh. Based on the Whttle s ndex polcy, LLLP can reduce the number of unfnshed EVs wth large remanng chargng demand and therefore reduce the non-completon penaltes. The total reward acheved by the Whttle s ndex wth LLLP nterchange polcy s more than twce of that obtaned by EDF; the performance gap between the the Whttle s ndex wth LLLP nterchange polcy and the LLF polcy s over 10%. We also note that the LLLP prncple mproves Whttle s ndex by over 10%. VI. CONCLUSION In ths paper, we consdered the problem of schedulng of the chargng of a large number EVs n publc facltes a problem of partcular potental sgnfcance as EV penetraton deepens. In such settngs, t s essental to develop hghly effcent and onlne chargng algorthms. To ths end, ndex polces consdered n ths paper are attractve for ts mplementaton smplcty and versatlty n ncorporatng varous operaton uncertantes. REFERENCES [1] Hybrdcars, plug-n vehcles monthly sales dashboard, Avalable from: dashboard/.

7 7 [2] U.S. Department of Energy, Electrc vechcle chargng staton locatons, Avalable from: locatons.html. [3] Z. Yu, S. L, and L. Tong, On Market Dynamcs of Electrc Vehcle Dffuson, n Proceedngs of the 52nd Annual Allerton Conferene on Communcaton, Control, and Computng, Oct [4] P. Whttle, Restless bandts: Actvty allocaton n a changng world, Journal of appled probablty, pp , [5] Y. Xu and F. Pan, Schedulng for chargng plug-n hybrd electrc vehcles, n Decson and Control (CDC), 2012 IEEE 51st Annual Conference on, pp , IEEE, [6] V. Raghunathan, V. Borkar, M. Cao, and P. R. Kumar, Index polces for real-tme multcast schedulng for wreless broadcast systems, n INFOCOM The 27th Conference on Computer Communcatons. IEEE, IEEE, [7] J. J. Jaramllo, R. Srkant, and L. Yng, Schedulng for optmal rate allocaton n ad hoc networks wth heterogeneous delay constrants, Selected Areas n Communcatons, IEEE Journal on, vol. 29, no. 5, pp , [8] S. S. Panwar, D. Towsley, and J. K. Wolf, Optmal Schedulng Polces for a class of Queues wth Customer Deadlnes to the Begnnng of Servce, Journal of Assocaton for Computng Machnery, vol. 35, pp , October. [9] D. Towsley and S. Panwar, On the Optmalty of Mnmum Laxty and Earlest Deadlne Schedulng for Real-Tme Multprocessors, n Proc. of IEEE Euromcro 90 Workshop on Real-Tme, pp , Jun [10] K. R. L. Kruk, J. Lehoczky and S. Shreve, Heavy traffc analyss for EDF queues wth renegng, Annals of Appled Probablty, vol. 21, no. 2, pp , [11] C. L. Lu and J. W. Layland, Schedulng algorthms for multprogrammng n a hard-real-tme envronment, Journal of ACM, vol. 20, pp , [12] M. Dertouzos, Control robotcs: the procedural control of physcal processes, n Proc. IFIP Congress, pp , [13] A. Mok, Fundamental desgn problmes of dstrbuted systems for the hard real-tme envronment. PhD thess, MIT, [14] M. L. Dertouzos and A. K. Mok, Multprocessor onlne schedulng of hard-real-tme tasks, IEEE Transactons on Software Engneerng, vol. 5, pp , [15] S. Chen, L. Tong, and T. He, Optmal deadlne schedulng wth commtment, n Communcaton, Control, and Computng (Allerton), th Annual Allerton Conference on, pp , IEEE, [16] S. Chen, Y. J, and L. Tong, Large scale chargng of electrc vehcles, n Power and Energy Socety General Meetng, 2012 IEEE, pp. 1 9, IEEE, [17] S. Chen and L. Tong, ems for large scale chargng of electrc vehcles: Archtecture and optmal onlne schedulng, n Smart Grd Communcatons (SmartGrdComm), 2012 IEEE Thrd Internatonal Conference on, pp , IEEE, [18] A. Subramanan, M. Garca, A. Domnguez-Garca, D. Callaway, K. Poolla, P. Varaya, et al., Real-tme schedulng of deferrable electrc loads, n Amercan Control Conference (ACC), 2012, pp , IEEE, [19] L. Gan, U. Topcu, and S. H. Low, Optmal decentralzed protocol for electrc vehcle chargng, Power Systems, IEEE Transactons on, vol. 28, no. 2, pp , [20] Z. Ma, D. S. Callaway, I. Hskens, et al., Decentralzed chargng control of large populatons of plug-n electrc vehcles, Control Systems Technology, IEEE Transactons on, vol. 21, no. 1, pp , [21] Q. L, T. Cu, R. Neg, F. Franchett, and M. D. Ilc, On-lne decentralzed chargng of plug-n electrc vehcles n power systems, arxv preprnt arxv: , [22] C. Wu, H. Mohsenan-Rad, and J. Huang, Vehcle-to-aggregator nteracton game, Smart Grd, IEEE Transactons on, vol. 3, no. 1, pp , [23] J. Donadee and M. D. Ilc, Stochastc optmzaton of grd to vehcle frequency regulaton capacty bds, Smart Grd, IEEE Transactons on, vol. 5, no. 2, pp , [24] F. Juul, M. Negrete-Pncetc, J. MacDonald, and D. Callaway, Realtme Schedulng of Electrc Vehcles for Ancllary Servces, n Proc IEEE Power and Energy Socety General Meetng, (Denver, CO), [25] E. Altman, Constraned Markov decson processes, vol. 7. CRC Press, [26] J. Nno-Mora et al., Restless bandts, partal conservaton laws and ndexablty, Advances n Appled Probablty, vol. 33, no. 1, pp , 2001.