Stackelberg Game-Theoretic Strategies for Virtual Power Plant and Associated Market Scheduling Under Smart Grid Communication Environment

In order to schedule the virtual power plant and corresponding energy market operation, a two-scenario Stackelberg game-theoretic model is proposed to describe interactions between market operator and VPP operator. During market operation, the market operator is a leader of the game to decide market cleaning prices, considering the power loss minimization of VPP operator, whereas during VPP operation, the VPP operator becomes a leader to facilitate the demand side management (DSM) through proper monetary compensation, considering the market trading balance between power sellers and power buyers. An optimal scheduling strategy including power dispatch and market balance will be realised. Case studies prove the effectiveness of the proposed Stackelberg game-theoretic model through IEEE 30-bus test system. The market scheduling promotes the power exchange among VPPs. The VPP scheduling evaluates the optimal monetary compensation rate to motivate the DSM including load shifting and load curtailment.


I. INTRODUCTION
T HE advantages of smart grids enable sophisticated com- munication technologies, sensing technologies, and control methodologies to be integrated into power systems [1].These technologies contribute to the improvement of smart control and management for the purpose of efficient operation of power newtworks and associated markets [2].Compared with traditional power systems, the smart grid system becomes complex with the introduction of renewable energy resources (DERs), distributed generators (DG), demand side management (DSM), and energy market strategies.Hence, designing a virtual power plant (VPP) to aggregate the capacities of diverse DERs and flexible demands becomes essential.The operations of power system and associated market are scheduled through those smart grid communication technologies.
Compared with the unit commitment problem, the conception of VPP pays more attention to market participation, which means that it is capable of involving more DERs into power networks scheduling and energy market trading [3].Hence, a scheduling strategy for both energy market and power system is required to help the distribution system operator (DSO) coordinate between market operator and VPP operator.This provides an opportunity for the application of game-theoretic model to analyse the interactions between market operator and VPP operator and optimize these scheduling problems.Depending on whether participants work together with common interests, the game theory consists of cooperation game and non-cooperation game.Cournot model and Stackelberg model are two typical models of non-cooperation game.In the Cournot model, each player makes its decision independently and simultaneously [4].However, during operations of power network and energy market, the VPP operator and market operator normally make decisions by sequential order, which is suitable for applying the Stackelberg game-theoretic model into scheduling process.The Stackelberg model features on two-level hierarchical decision-making process.The leader in the first level announces its strategy in the first place, and has an idea about the responding action of follower.The follower in the second level subsequently makes responsive strategy to reply to the leader.Received this responsive strategy, the leader finalizes its strategy.Involving the Stackelberg game theoretic model into both market scheduling and VPP scheduling, however, has seldom been studied.
With respect to power generation scheduling, the day-ahead market financially schedules biding and offering for balancing energy supply and demand one day before system settlement.A sophisticated day-ahead scheduling pursues the power system reliability and fair market trading.The reliability can be realised through optimizing power flow and delivering the DSM as proposed in [5]- [7].Meanwhile, a fair market trading can be guaranteed by considering the interests in both power buyers and power sellers.A stochastic environmental and economic dispatch of VPP was proposed in [8] for generation scheduling, but the market operation was not considered.The research in [9] sought to strike a balance among carbon reduction, payment bills, and costs through multiobjective optimization scheduling.Based on these researches, price signals formulated by market operator and power allocation performed by VPP operator are further investigated to explore mechanisms of system coordination.Both of these functions build on smart grid communication technologies and attribute to the market scheduling and VPP scheduling through Stack-elberg game-theoretic models.The market operator and VPP operator are included into the DSO.
Therefore, a dedicated study regarding VPP and market scheduling will be conducted through two-scenario Stackelberg game-theoretic model, under the circumstance of smart grid communication.The proposed model contributes to attract and optimise DERs to participate in power grids scheduling and energy market trading.On one hand, different from the conventional unit commitment problem, in which schedules are limited by the coverage area and reliability due to regional integration of microgrid, VPP processes the scheduling and makes decisions through central control system and information agent unit.Thus, the VPP breaks geographic restrictions to transmit power and minimize power loss.The supplydemand balance can therefore be guaranteed for the whole power networks.On the other hand, with the optimization of the costs for power sellers and the payment bills for power buyers, the economic benefit can be improved and reallocated to participants of energy market.
Building on existing work, contributions of this research are as follows:1) Instead of scheduling the VPP output in a single model, the market operation and VPP operation are separated into two-scenario Stackelberg game-theoretic strategies, according to different functions performed by market operator and VPP operator; 2) Regional difference of power prices and time difference of monetary compensation rates for DSM are considered depending on VPP output , so that market mechanism can be applied into power trading among VPPs.
The remaining paper is organized as follows.Section II formulates the two-scenario Stackelberg-game theoretic framework and the inside components of the VPP.Section III and Section IV mathematically describe the game-theoretic model for market scheduling and VPP scheduling, respectively.Section V presents the results of case studies for daily VPP scheduling.Finally, Section VI draws the conclusion.

II. SYSTEM FRAMEWORK
This section illustrates the system model of two-scenario Stackelberg game-theoretic scheduling for market operation and VPP operation.The inside components of VPP and power trading mechanism are also introduced.

A. Two-Scenario Stackelberg Game-theoretic Framework
Intelligent communication technology and software architecture enable the globally DERs and flexible demand to be aggregated and optimized as a VPP [10].The VPP is subsequently conceptualized as a real power plant to participate in both power and market schedules.A two-scenario Stackelberg game-theoretic model is formulated for VPP schedules as presented in Fig. 1.This model is performed hourly by the DSO in day-ahead energy market to schedule both market operation and power allocation.The corresponding schedules are managed by market operator and VPP operator, respectively.During market scheduling, the leader (market operator) forecasts the power demand for next 24 hours.This forecasting information enables the follower (VPP operator) to perform power flow analysis and find a proper generation dispatch for each VPP with minimal power loss.With this power generation dispatch, VPPs provide their bids to the wholesale market to sell extra power based on market operation mechanisms.The market cleaning price is subsequently decided in balancing supply and demand.
During the VPP scheduling, VPP operator and market operator exchange their roles, which means that the VPP operator becomes a leader to take the first mover's advantage [11].To help stimulate the DSM scheme, the VPP operator formulates a set of hourly monetary compensation rates for consumers to reshape their consumption behaviours.The market operator subsequently balances the interests for the participants of energy market, i.e. minimizing costs of power sellers and payments of power buyers.VPP operator therefore finalises the power dispatch in supply and demand sides.

B. Virtual Power Plant Components
Advanced smart meter enables bidirectional communications to be realised between consumers and power grids, which supports the communication infrastructure of the VPP [12].Hence, the VPP gathers scattered DSM, DG, and electric vehicle (EV) to coordinate and optimize aggregated power output, because the scattering distribution of consumers limits their negotiation power in energy markets.The DSO also faces challenges of managing the large-scale consumers.This research considers the DSM, DG, and EV as components of VPP.For the DSM, the incentive signal or price signal help the realisation of load curtailment or load shifting [13].The VPP is capable of gathering scattered distributed DSM resources by proper contracts.The consumers can conversely sell the DSM output as the VPP output [14].Similarly, the VPP organises the DGs, and sells the extra output to other VPPs through microgrids [15].Furthermore, the communication infrastructure of the VPP supports a platform between EV users and microgrids.The EV is not only taken as a power load when it needs to be charged from microgrids [16], but taken as a storage system when the extra power sells back to the microgrids.

III. GAME THEORY FOR MARKET SCHEDULING
During the market scheduling, interactions between market operator and VPP operator are considered to provide a sequential game formulation including two players.Firstly, the leader (market operator) announces leader's strategy i.e. the prediction of hourly demand over next 24 hours.Secondly, received this strategy, the follower (VPP operator) minimizes the power loss of power networks and maintains technical constraints.They schedule the operation of VPPs through the decision variables of power dispatch.Thirdly, according to the follower's responses, the leader obtains the power driving pricing vector, before finalizing the schedules based on market cleaning mechanisms.This game theory model is designed to run in one hour interval to optimise the market scheduling.

A. Power Driving Pricing
Considering a power network consisting of N power buses, indexed by integer i, i = 1, ..., N .Each VPP covers a certain amount of buses and DG to supply the power demand or perform the DSM.In our model, there are M VPPs, indexed by j, j = 1...M .Let load(j) denote the power load that belongs to the jth VPP, and gen(j) denote the generator that belongs to the jth VPP.Define P gen j and P load j are the total power generation and consumption of jth VPP, respectively.The power is generated by the DG within each VPP, which can be described as: where P G i is the power generation of bus i at hour t.
Similarly, the power is consumed by the load connected to that bus, which can be described as: where P L i is the power consumption in bus i at hour t.Therefore, the power ejection of jth VPP is: where P j (t) is power ejection of jth VPP at hour t.
During the day-ahead market, when the power generation exceeds the demand in one VPP, i.e. the VPP ejection is positive, this VPP can sell the power on power driving price to other VPPs, in which power supply cannot support the demand.Those VPP sellers announce their bids to form the price signal vector over next 24 hours, defined by price(t), which is dependent on the power ejection of each VPP seller.Similar with [17], in which the unit energy price at each bus is set as a linear function of power difference between generation and consumption, the power price at the jth VPP is modelled as a linear function of P j (t): Furthermore, according to the operational principle of the energy market, when the demand of other VPPs is met by the supply of total j VPPs, the price of jth VPP becomes a market cleaning price of all the VPPs to sell their power.

B. Objective of Virtual Power Plant Operator
The power buses are interconnected with each other to form the grid topology.The power and corresponding loss are transmitted among buses through branches.Define E pq is the power ejection from bus p to bus q over branch pq and I pq is the power injection from bus p to bus q over branch pq, p, q = 1, ...N .Hence, the power loss caused by power transmission can be described as the difference between power ejection and injection over each branch: where P loss pq (t) is power loss over branch pq at hour t.
The power flow distribution can be obtained, after power flow analysis is performed by Matpower [18].Therefore, objective function of VPP operator is to minimize the power loss of power networks:

C. Solution of Market Scheduling
The mathematical presentation of market scheduling is a non-linear programming problem due to the power flow calculation.Hence, the conventional convex optimization solution is unable to solve it.We design a solution by applying the particle swarm optimization (PSO) algorithm [19] during the power loss minimization.The steps of solving this problem are described as follows.
Input: The IEEE bus test system, total VPP number:M , total bus number:N , P load j , P gen j , population size P OP size Step 1: For each particle m ∈ P OP size , initialize the velocity v m and position x m with a uniformly distributed random where b lo and b up are the lower and upper limits of the searching space, x min and x max are minimum and maximum ranges of position, and v min and v max are minimum and maximum ranges of velocity.
Step 2: Let P G i = x m , run power flow analysis through Matpower for each x m to obtain E pq and I pq , before calculating P loss pq through (6).
Step 3: Let pBest m be particle m's best known position and let gBest be the entire swarm's best known position.Based on step 2, evaluate the fitness of particle m and set pBest m = x m to find the optimal solution with minimal power loss.
Step 5: For each particle m ∈ P OP size , select random numbers r p , r g ∼ U (0, 1) to update the velocity v m by (7) and position x m by x m ← x m + v m , until the termination criterion is met.(7) where c 1 and c 2 describe how much a particle trusts its personal attractor and global attractor, respectively.
Step 6: Let P G i = gBest, allocate buses to corresponding VPPs, before calculating the power ejection of each VPP P j by (3) and the price vector price j by (4).
Step 7: Sort the price vector price j in sequence order, and match them with each VPP ejection vector P j .While j M P j = f demand , P rice j = P rice * , where f demand is the total power demand of other power plan, and P rice * is the market cleaning price for the VPP power exchange.
Output: Power price P rice * , and Power dispatch P j .

IV. GAME THEORY FOR VIRTUAL POWER PLANT SCHEDULING
The VPP scheduling aims to allocate optimised VPP output considering the DSM.Hence, the VPP operator becomes the leader to take the first mover's advantage of the Stackelberg game-theoretic model [20].First, the leader (VPP operator) announces a leader's strategy, i.e. the monetary compensation rate for applying the DSM.Secondly, with this compensation information, the follower (market operator) seeks to optimize the interests for the market participants: power buyers and power sellers.For the power sellers, the objective of them is to minimize the operating cost of VPP.By contrast, for the power buyers, the objective of them is to minimize the total payment purchasing for the needed power.These two opposing objectives lead to a multiobjective optimization problem (MOP) for market operator.

A. Demand Side Management
The designed DSM technique including load shifting and load curtailment schedules connecting moments of loads in consumption side to realise objective demand curve.Load shifting seeks to optimise the usage period of shiftable load, because the running of them can be shifted from high-price period to low price-period remaining the total energy consumption unchanged.By contrast, load curtailment dynamically decreases the consumption level of curtailable load.Customers of VPP are assumed to be price-sensitive to participate the DSM scheme with a proper monetary compensation.The change of load consumption at each hour t = 1, 2, ..., T is modelled as a linear function of monetary compensation.
where f c (t) is the shifted or curtailed load at each hour t through DSM, compe(t) is monetary compensation in each hour t, and γ and δ are DSM coefficients.
The load shifting technique of DSM controls the time period of appliances connecting.During off-peak demand period, compe(t) ≥ 0, because the load consumption would be shifted to this period with incentive of monetary compensation.Additionally, in order to remain the total consumption level of shiftable appliances the same, we have: The load curtailment is conducted to reduce the total power consumption when it is necessary.The consumers subsequently receive the monetary compensation for the inconvenience.The maximum level of load curtailment is set considering the interests and acceptance level of customers: 0 ≤ f c (t) ≤ f max curt , where f max curt is the maximum level of load curtailment.

B. Multiobjective Optimization Problem of Market Operator
1) Objective of Power Sellers: The objective of power sellers is to minimize their cost.Cost functions are defined to describe the cost of power ejection from VPP components: Demand Side Management: where Cost DSMj (t) is overall cost of deploying DSM in the jth VPP at time t, f cj (t) is the amount of load shifting or load curtailment through DSM in the jth VPP at time t, and a DSM , b DSM , and c DSM are cost coefficients of DSM unit.Distributed Generation: where Cost DGj (t) is the overall cost for DG operation in the jth VPP at time t, DG j (t) is the power output of DG in the jth VPP which is sold back to the microgrids at time t, and a DG , b DG , and c DG are the cost coefficients of DG.There is also a limitation for the maximal output of DG: where DG max j is maximum power output of DG in jth VPP.Electric Vehicle: where Cost EVj (t) is the overall cost of EV in the jth VPP at time t, EV j (t) is the power output of EV in the jth VPP at time t, and a EV , b EV , and c EV are cost coefficients of EV.Considering the capacity limitation of microgrids, the maximum power output of EV is set: where EV max j is the maximum power output of EV.Thus, the objective function of power sellers is: Objective of cost minimization : (14) 2) Objective of Power Buyers: The objective of power buyers is to minimize their payment bills for the power needed, which can be described as the bill for power demand minus the bill-saving due to the load shifting and load curtailment: Objective of payment bill minimization :  16)

C. Solution of Virtual Power Plant Scheduling
To solve the VPP scheduling problem, we introduce a multiobjective immune algorithm (MOIA) to solve the MOP included Stackelberg Game-theoretic as shown in Algorithm 1. Detailed MOIA is presented in previous research [21].Conduct evaluation for the rest antibodies through optimization constraints, before removing infeasible antibodies.

7:
if iteration population size exceeds the required size then 8: Reduce population size for antibodies normalization.

V. CASE STUDIES
To demonstrate the performance and effectiveness of the proposed two-scenario Stackelberg game-theoretic model, case studies are conducted.The U.K. power supply and demand data on the average output of all forms of generations is adopted from Gridwatch [22] and applied into the IEEE 30 bus-test system through replacing the system data by proportion [23].This system consists of 6 generators, 30 buses including 21 buses with loads, and 41 branches [24].Every 5 buses are allocated in one VPP in sequence order.There are total 6 VPPs.With respect to the VPP components, the maximum of 5% of load curtailment is assumed through price incentive.The load curtailment is conducted during peak demand period between 16 h and 22 h.The cost coefficients are adopted from the U.K. average cost for projects commissioning in 2016 [25].

A. Game Theory for Market Scheduling
The VPP 1, VPP 3, and VPP 4 generate extra power to sell in the energy market as power sellers, after meeting their own power demand.By contrast, the VPP 2, VPP 5, and VPP   6 need to buy power from the wholesale market as power buyers.The results of daily market scheduling is presented in Fig. 2. It is clear that the VPP 4 becomes the marginal sellers to decide the market cleaning price at most of time.Besides, it is worth mentioning that during the 1, 10, 11, and 18 hours, the total output of VPPs is unable to meet the total demand.Therefore, they have to import the power from the main grids.Furthermore, the comparison between original hourly power loss and power loss after scheduling for the IEEE 30 bus-test system is illustrated in Fig. 3. Through the market scheduling, the total power loss has been reduced from 12.12 M W to 11.75 M W , reducing by 3.05 %.The power loss during the peak demand period decreases more dramatically than that during the off-peak demand period.This is because the power loss is primarily driven by power demand.

B. Game Theory for Virtual Power Plant Scheduling
The second scenario is performed for VPP scheduling.The results of daily VPP scheduling in one hour interval are presented in Fig. 4. It is clear that with the proper monetary compensation rate, the power consumption in peak demand period (16 h to 22 h) is shifted to the off-peak demand period (1 h to 16 h and 22 h to 24 h).The total 0.73 M W of load curtailment is also realised.With respect to the specific components inside the VPPs, the total daily DG output for power trading among VPPs reaches 54.79 M W , and the total daily EV output for power trading among VPPs reaches 41.46 M W .After VPP scheduling, these power exchanges create total 36.98GBP profits for power sellers, compared with 16.22 GBP before scheduling.Thus, the VPP scheduling creates the additional economic benefit in energy market.The comparison of optimization objectives including payment bills for power buyers and costs for power sellers is shown in Fig. 5.With the almost unchanged payment bills, the generation costs are reduced.It is particularly for the peak demand period, when 7 GBP/ h cost-saving is realised.

VI. CONCLUSION
This paper proposes a two-scenario Stackelberg gametheoretic model for both energy market and power system operations.The first level Stackelberg game-theoretic model for market scheduling manages the power trading between VPP sellers and VPP buyers, which enables the supply-demand balance to be maintained on a proper market cleaning price for power trading.The market scheduling also contributes to the reduction of power loss by 3.05 % to ensure system reliability.Besides, the second scenario Stackelberg gametheoretic model for VPP scheduling helps to motivate proposed DSM scheme with optimal monetary compensation rate.0.73 MW load curtailment during peak demand period is realised.

10 :
until Reach the maximum time of iteration.Output: Power allocation p load j , and p gen j .

Fig. 3 .
Fig. 3.The comparison of power loss before and after scheduling.

Fig. 4 .
Fig.4.Results of the virtual power plant scheduling.

Fig. 5 .
Fig. 5.The results of optimization objectives compared with original objectives.