Optimal team communication structures in microgrids

This paper presents a team player based modeling approach to communication structures in microgrids. The modeling follows a game theoretic framework by forming teams inside the Small Scale Power Systems (SSPS). The team players are able to minimize the common objective when there is communication, and shift to the individual objectives when communication fails. The paper also presents analysis to determine the minimal performance standards for the given level of communicated information. The last part of the paper shows how the Stackelberg concept can be integrated to the micro-grid for a leader-follower modeling approach. Example systems are simulated to explain each case.


I. INTRODUCTION
An small-scale power system (SSPS) is an integrated power network which delivers and absorbs electric energy. In most cases, the loads and sources in SSPS have highly nonlinear and multi-objectives, while the power generation must always match demand. Further, a change of a load or source will influence the real time system parameters of the system given the low inertia of generation. Micro-grids [1]- [3], Naval ship electric power systems (NSEPS) [4], spacecraft systems [5], [6], telecommunication power systems [7], and automotive power systems [8], [9] are typical examples of SSPS. Higher reliability, controllability, ease distributed generation and penetration of renewable generations [1], [10], [11] are all reasons SSPS are becoming an attractive alternative to the more traditional large-scale utility grid.
In large-scale systems, the loads are usually modeled as constant power, or PQ busses, and PV buses are used to model constant generation [12]. In this construct, the power flow problem must be numerically solved as described in [13] under the general assumption that generation matches demand [12]. However, in a SSPS, the load and sources can be modeled as dynamic energy resources since they are controlable power electronic interfaces (PEI) [14]. In a game theoretic modeling approach, these dynamic quantities can be called players in the system. Then, the control system should provide the required flexibility of the players to ensure operation as a single aggregated system. The modeling discussed is based on micro-grids but, the discussed methods can be easily applied to other types of SSPS as well.
In electric power systems, there is always some form of finite energy in the system. However, in micro-grids the energy availability and stability margins are closer to their operating and equilibrium points. At this point, neither player wants to change their local objective and maintain at least a Pareto optimal equilibrium. Further, if a considered player tries to deviate from this operating point to improve its local objective, this causes loss in other players and removes the equilibrium from the system. However, in most cases, this equilibrium is not the global equilibrium of individual players.
Many techniques have been reported in the literature to study system dynamics. However, the studies related to analyzing micro-grids using game theory is limited. Game theory is a branch in applied mathematics that has been used in social science, economics, engineering, politics, computer science, and philosophy. Game theory captures the nature of strategic situations, or games, in which an individual's successful choices depends on the choices of others on a common system [15]- [17]. Application of game-theory can be seen in engineering in the areas of power system [12], [18], [19], electricity market financial transactions [20]- [24], traffic planning [25], [26], decision making [27], manufacturing [28], computer networks [29], [30], and communication networks [31]. In [12] the modeling procedure for a PEI as a dynamic, controllable energy resource is shown. Following this approach, the methodology to obtain the systems equilibrium under different converter objectives is given. The systems Nash equilibrium (NE), is obtained in modeling and provides an operating point to the player. This paper presents a method to improve the converters local operating point by forming teams within the network. The team player participation in game theory is discussed in [16], [17]. However, application of these methodologies in micro-grids has not been given much attention in the literature. This approach is advantageous since micro-grids have limited resources which need to be allocated effectively. In most cases it is possible to form teams in a micro-grid under dynamic operation. A team can be formed considering loads which operate close to each other or sources which can communicate with each other.
The proposed improved method is based on a communication structure between team players. It takes into account the required information level and the corresponding team player improvement in the micro-grid. However, having more infirmation implies that more communication channels are available in the network. This causes complexity, increased cost and increased failure modes in the system. Therefore, there is always a trade off between level of improvement that a player archives vs increased communication. This paper finds the new equilibrium of team players with player by player(pbp) optimization which can be used to build distributed controllers for the converter. In this way the initial non cooperative game is transformed in to possible cooperative game sets, which show improvements in objectives over noncommunication. It address the issues of how effective the initial Nash strategy is compared to the different team optimal strategies. In each method, it is given more attention to reduce the level of information necessary to achieve improvement. In addition, it shows the optimum information mixing of the players. In this way, players can operate in their new equilibrium while communication exists. In any instance, if communication fails, then they can switch to the prior operating point while maintaining a minimal level of Pareto optimality. The last section of the paper is focused on the Stackelberg concept based modeling in a micro-grid [16]. This helps to define teams in micro-grid with leader-follower approach. Under this modeling, a considered leading player achieves a better operating point than the team equilibrium. However, this improvement causes the followers to get weaker operating points than the team equilibrium.

II. PLAYER OBJECTIVES IN A MICRO-GRID
In the most generic case, any power system with a PEI can be simplified and viewed as the translator between the power system and the load. However, this PEI can be modeled as a controllable dynamic quantity as discussed in [12]. Therefore the PEI becomes a player in the system. Consider an microgrid with V and I as the bus bar voltage and injected current vectors. If Y represents the bus admittance matrix of this micro-grid, then the relationship between these two vectors can be written as All vectors are of dimension m × 1, with m representing the number of bus bars. In general, the bus admittance matrix Y is symmetric and is a function of line admittances and shunt load resistances of the system. The process of solving nodal equations in (1) and the bus bar voltage calculation is discussed in [12], [13]. The player objective function can be modeled as the general form where r −i are the players other than r i . The Nash equilibrium [12], [16], [17] of player's is found at a point whereby no player has incentive to change its current control decision such that In other words, at NE player has minimized its cost function under the all other player reactions. Such a equilibrium point the player objective is J i (r * i , r * −i ) and optimal player control r * i decision is given by

III. TEAM MODELING
The NE discussed in section II may or may not be the global minimum of a player. In a micro-grid it is possible to form teams among players. Normally team players have similar requirements within the system and may be physically closer. For example, three or four loads operating near each other can form a team within the micro-grid. Otherwise, source players can make a team in side the micro-grid. It is necessary to study the possibility of improving the equilibrium through communication between team players. Under each modification, the corresponding minimization function of the players will be changed. Therefore, it is necessary to develop a method to determine the existence of the NE of the modified game.
Theorem: For each i ∈ N , let U i be a closed, bounded and convex subset of a finite-dimensional Euclidean space, and the cost functional J i : U i × ... × U N −→ R be jointly continuous in all its arguments and strictly convex in u i for every u j ∈ U j , j ∈ N ,j = i. Then, the associated N -person nonzero-sum game admits a Nash equilibrium in pure strategies [16].
It is possible to use the above mentioned theory to determine the existence of NE of modified game. However, checking the convexity of highly nonlinear terms in power system equations is a complicated task. Therefore, in such a situation, it is necessary to carry out simulations under turn based approach, and the system needs to obtain equilibrium. However, it does not grantee the new equilibrium is improved over the noncooperative game. In such situation by obtaining player's local objectives under the new equilibrium, the improvement of the objective can be checked.
If there exist a communication channel among all players in the team, then they can minimize a common objective function which may give lower value for the local objective. However, when the communication fails, they can switch back to the local objectives.One possible common objective function which can be used is shown in (4) [16]. Where p is the number of players in the team and w i 's are the weighting factors.
This common objective has two equilibrium points [16]. However, the distributed controllers need to consider player by player optimality. The existence of equilibrium can be decide based on methods discussed above. The equilibrium point will be such that Again, obtaining the solution of the highly nonlinear power system equation of type (5) is complicated. In such situations, it is necessary to obtain the solution using the turn based approach. In addition, availability of many players in the team needs a higher level of communication among players under common objective. Therefore, the team objective function is modified as in (6) to minimize the communication needed per player. Consider the team player objective in (4) with w i 's equal to 1. According to the above discussion, J G is a team objective of a team . Which means α 1 J G + α 2 J G is also a team objective for the considered team. α i 's are scalers, can be defined as mixing factor of converter objectives. However, shows that there exist another person by person game that defines the local objective as depicted in Fig.1. This modeling reduces information required per player to only neighboring information. In such situations, players operating points will be shifted to a point where To achieve an improvement, it is necessary to have a lower cost of team player's local objectives under the new equilibrium. This calculation is possible since the turn based approach evaluates the player's new equilibrium.

IV. STAKELBERG MODEL
In some cases, of team modeling it is necessary to give priority to one player within a converter team by defining a leader. Based on the choice of leader, remaining controllers need to take their control actions, act like followers. This modeling is important especially in a micro-grid with high sensitive/higher priority loads. The Stackelberg concept helps to define a team in such an approach. Literature shows that the Stackelberg concept based modeling can be split into three different categories within the framework of noncooperative decision making [16]. For example, consider three-person nonzero-sum games. If i th player (P i ) cost function J i (r 1 , r 2 , r 3 ) and corresponding control strategies are {r 1 ∈ L 1 , r 2 ∈ L 2 , r 3 ∈ L 3 }. Where L i denotes the control strategy sets.
Hierarchy 1: one leader and two followers. The followers react to the leader's announced control strategy . They play according to a specific equilibrium concept among themselves (for instance Nash). In such situations the controller finds the equilibrium such that [16] max r2,r3∈L F (r * 1 ) where L F (r * 1 ) is the optimal response set of the follower's group and is defined for each r 1 ∈ L 1 by

Hierarchy 2:
The second method is two leaders and one follower. The leaders play according to a specific equilibrium concept among themselves. He takes into account possible optimal responses of the follower.
Hierarchy 3: First player one (P 1 ) announces his control strategy.Then player two (P 2 ) determines his strategy by also taking into account possible responses of player three (P 3 ) and enforces this strategy on him. Finally P 3 optimizes his objective in view of the announced strategies of P 1 and P 2 . The Equilibrium definitions for Hierarchy 2 and Hierarchy 3 can be easily build same as in Hierarchy 1. Any way, those definitions are not included in this in order to reduce complexity.

A. Load player Teams Participation
Consider the nine bus system shown in Fig.2, based on the structure of the Electric Power Research Institute (EPRI)WSCC test case [12], [14]. In this paper, it is assumed that all the line resistances are 0.087 pu . This system has three load players load 5, load 6, load 8 and three source players . It is possible to form teams as discussed above either considering loads or sources.
First the effect of load teams in the system considered. The source voltages fixed at 1.05 pu. The individual load player minimization objective was chosen as in (8). This means the converter wants to regulate its input power while maximizing its terminal voltage. The desired power level of the player and weights used in each simulation are shown in Table I. The Fig. 2. Nine-bus dc power system with two dynamic loads [12] load team was defined with loads at bus bar 5, 6 and 8 as player 1, 2 and 3.
The observed equilibrium and player objectives under above discussed approaches are shown in Table II. According to  Table II, having more information helps to improve the load player local objectives. In addition, the team player equilibrium points also increase under this objective. In this example, the information required for two methods are one information and two information. Those are neighboring player local objectives and remaining team player local objectives.

B. Source Player Teams Participation
The system was modified to observe the effects of the source team participation. In this modeling, the load converter control strategy was kept constat and source converters change their control strategies to minimize objectives . The source player minimization objective was chosen as in (9). Then following sets can be defined. N ={1,2,3,5,6,8}, S={1,2,3}, L={5,6,8} and let , V={4,7,9}. In addition v reg was considered as 1 pu for all source players.
where β i denotes the weighting factors of source player objectives. The load 5, 6 and 8 were fixed at 0.95 pu, 0.88 pu and 0.75 pu. The constant used under simulation is shown in Table III. The observed results are shown in Table IV under each cooperative game.
According to the results in Table IV, that source player can improve their objectives with communication. In this particular example, having one information improves the objectives 25% from the noncooperative level. However, it shows only 2-6% improvement from one information level to all information level.

C. Load and Source Team Participation
The effects of forming a load team and a source team simultaneously in the system were considered. The constants used for source players are as in Table III and load player constants as in Table I. However, this case simulation was limited to one information improvement.
According to the results in Table V, in this case there is a slight deviation of player equilibrium from the individual team participation cases. Therefore, it can be concluded that the single team participation and simultaneous participation has effects in power system game with communication. This will changed with the different player objectives and control input selections.

VI. ALTERNATIVE OBJECTIVES
In the preceding sections, the load player game was to regulate power but also maximizing terminal voltage. However, in some cases players may interest of maximizing their power. This scenario exists in cases when system experiences a disturbances or excessive loss of generation. In such situations, players objectives will switch to a function as in (10) Then the objectives were improved using the communications approaches discussed above. The constants used as α 1 = 1 and α 2 = 1, w 1 = 1, w 2 = 1, and w 3 = 1.01. The observed results are shown in Table VI.  The players reach an equal equilibrium point of 0.071 pu with equal local objectives of -1.9279 with no information. Since the network is symmetric through bus bar 8, players reach equal equilibrium point and cost. In this case, it shows 7% improvement in local objectives having one information. Obtaining all information it can be improved by 2.2% from the one information level. However, in this case information mixing shows significant sensitivities with the local objective. Therefore, a sensitivity analysis was carried out for player local objective with α 2 while α 1 = 1. The sensitivity analysis helps to determine the optimal information mixing of the players and the observed results are shown in Fig.3.
In Fig.3 it is seen that the player local objectives can achieve the minimum of entire minimum set at α 2 = 2.01, while α 1 = 1. Therefore, in this case, the better option is to get 2.01

VII. STAKELBERG EXAMPLE
The objective improvement observed in each model discussed above does not help a given player to reach higher levels than his team members. Therefore the above discussed load team was simulated under Stackleberg modeling with the following hierarchies under the objective in (10). In each method given the player P 5 controls strategy space is chosen randomly from 0.001 pu to 2.001 pu.
Hierarchy 1 (H1): Player 5 (P 5 ) is the leader and players 6 and 8 (P 6 , P 8 ) are followers. P 6 and P 8 react to the leader's announced control strategy given in (11) Hierarchy 2 (H2): P 5 and P 8 are the leaders of the team. P 6 will act as a follower. P 6 reacts according to the leaders announced control strategy given in (12) Hierarchy 3 (H3): P 1 announces his control strategy. Player P 2 then determines his strategy by taking the response of P 1 . P 2 then affirms his strategy to P 3 . Finally, P 3 optimizes his objective in view of the announced strategies of P 1 and P 2 as in (13) In each hierarchy the observed results are shown in Table  VII. The team was limited to load team in the micro-grid under this concept. According to Table VII, under H1, the leader achieves highest possible outcome under limited resources. The followers had to share the remaining. In this particular example the leader achieves a -3.217 cost. The followers achieve their goals equally of -1.0699 according to the response of the leader. In H2, leaders P 5 and P 8 achieve their targets equally of -2.3619 in this example. The follower P 6 has to use the remaining resources according to the announced strategies of leaders and reaches the P 6 objective of -0.9883. Finally in H3, players have different levels like the main leader and the sub leader in the team. As a result the main leader achieves the highest objective of -2.8618 while sub leader shows a cost of -1.4861. The follower gets the remaining -1.203. In each Stackleberg concept modeling the necessary information from remaining players is the control input. If it is necessary this modeling can be modified to the model based on the remaining player objective information. The Stackleberg concept based models can be built in different ways depending on the requirements in the micro-grid. However, this example is limited only to show the possibility of this method used in the micro-grid for different objectives.

VIII. CONCLUSIONS
The formulation of team players in micro-grids helps to improve the static operating point in the system. The feasible level of improvement depends on the level of communication available. In each method, players forms teams and modify the minimization objectives. In addition, the individual team participation and simultaneous team player participation in micro-grid shows different local objectives. The direct solution for equilibrium for highly nonlinear power system equations is difficult. The turn based approach improves the solution to obtain the equilibrium under each team model of converter teams. The improved model using neighboring information is important to reduce the required information level. However, during information mixing a significant sensitivity for player local objectives is seen. Therefore, to achieve optimum improvement it is necessary to mix information under the optimum value. The Stackleberg concept helps to allocate bigger portions of resources for given players depending on requirements. Expanding this analysis for different converter objectives, multiple team participation, different methods of objective mixing are possible future directions of this project. In addition it is necessary to observe the sensitivity of each mixing factor under each model. Further, different formulation methods for each hierarchy of Stackleberg concepts are also interesting future directions.