Decarbonization of Electricity Systems in Europe: Market Design Challenges

Driven by climate change concerns, Europe has taken significant initiatives toward the decarbonization of its energy system. The European Commission (EC) has set targets for 2030 to achieve at least 40% reduction in greenhouse gas emissions with respect to the 1990 baseline level and cover at least 32% of the total energy consumption in the European Union (EU) through renewable energy sources, predominantly wind and solar generation. However, these technologies are inherently characterized by high variability, limited predictability and controllability, and lack of inertia, significantly increasing the balancing requirements of the system with respect to historical levels. The flexibility burden is currently carried by flexible fossil-fueled conventional generators (mainly gas), which are required to produce significantly less energy (as low operating cost and CO2-free renewable and nuclear generation are prioritized in the merit order) and operate part loaded with frequent startup and shut-down cycles, with devastating effects on their cost efficiency.

DRIVEN BY CLIMATE CHANGE concerns, Europe has taken significant initiatives toward the decarbonization of its energy system. The European Commission (EC) has set targets for 2030 to achieve at least 40% reduction in greenhouse gas emissions with respect to the 1990 baseline level and cover at least 32% of the total energy consumption in the European Union (EU) through renewable energy sources, predominantly wind and solar generation. However, these technologies are inherently characterized by high variability, limited predictability and controllability, and lack of inertia, significantly increasing the balancing requirements of the system with respect to historical levels. The flexibility burden is currently carried by flexible fossil-fueled conventional generators (mainly gas), which are required to produce significantly less energy (as low operating cost and CO 2 -free renewable and nuclear generation are prioritized in the merit order) and operate part loaded with frequent startup and shut-down cycles, with devastating effects on their cost efficiency.
Furthermore, the decarbonization agenda is also envisaged to affect the demand side, mainly through the electrification of segments of the transport, heating, and cooling sectors that are currently heavily reliant on fossil fuels. However, this electrification is expected to yield a disproportionately higher increase in peak electricity demand levels than the associated increase in the overall electrical energy consumption due to the temporal patterns in the usage of vehicles and heating/cooling appliances. This implies that capital-intensive investments in new generation capacity and network reinforcements will be required, and this new infrastructure will be significantly underutilized. Considering these challenges, as the decarbonization initiatives further develop, the utilization of the generation and network infrastructure is constantly reducing, and the total electricity system costs are dramatically increasing.
Beyond the technical challenges associated with increasing balancing requirements and peak demand levels driven by the decarbonization of the European energy system, there are growing challenges associated with the design of electricity markets. The key market challenges include a) the "merit-order effect" of renewable generation and the resulting "missing money" problem faced by the generation side; b) the integration of variable renewable generation in energy and ancillary services markets; c) the design of effective carbon emissions markets; d) the capture of the full system value of distributed flexibility in energy and balancing markets; and e) the geographical integration of different market segments, including the development of a harmonized pan-European market and the coordination of emerging local energy markets. This article aims at providing evidence of these challenges in the European setting, reviewing European policy initiatives to address them, and identifying open issues toward developing innovative electricity market designs to enable the cost-effective and secure development of a highly decarbonized European electricity system.

The Need for a Radical Change of Electricity Market Design
Beyond setting ambitious carbon reduction targets, the EC remains committed to a deregulated electricity market paradigm, according to which the investment and operation of the generation, demand, and energy storage components are driven by competitive markets encapsulating profit-driven market participants. This implies that both the large-scale integration of low-carbon generation as well as the realization of the system benefits of flexibility resources will have a significant impact on the current market dynamics and require a fundamentally new market design.
The most fundamental feature of this new market design lies in shifting the focus from the operation timescale and the shortrun marginal cost of the system toward the investment timescale and the necessary capital investments to support the decarbonization agenda, as qualitatively illustrated in Figure 1. Under a largescale integration of renewables, the short-run marginal cost and consequently the prices in the energy market will be massively reduced due to the very low (nearly zero) marginal production costs of these resources; this is widely known as the merit-order effect of renewables. On the other hand, the value and prices of ancillary services will be increased by an order of magnitude, mainly due to the higher balancing requirements driven by the variability of renewables. Despite the controversy around capacity markets, their size is also expected to grow in the next decade due to the increasing need to remunerate and recover the investment costs of conventional generation, preserving the security of supply. More importantly, a large increase in the total low-carbon generation investment costs is expected (despite the reducing unit costs of renewables) to achieve the ambitious carbon reduction targets. Furthermore, the decarbonization of end-demand segments (e.g., heating, transport, and industry) will also require significant investment, and appropriate energy policy initiatives will need to be developed.

Merit-Order Effect in the European Electricity System
Various studies have recently investigated the merit-order effect of renewable generation in European markets. Two representative examples are presented in this article. The first one focused on the Portuguese day-ahead prices and was conducted by the National Laboratory of Energy and Geology. The time period of the study ranged from 1 January to 30 June 2016 (a total of 4,368 h). The simulations considered data extracted from the Iberian (Spanish and Portuguese) electricity market and were performed with the agent-based simulation tool, MATREM (for Multi-Agent Trading in Energy Markets). The results indicate that a wind penetration of 28.1% in the Portuguese system yielded an average price reduction of about 17 €/MWh during the first half of 2016. The highest reduction in the study period, about 25 €/MWh, was observed in January, which was a particularly windy month. The merit-order effect for one time period (2 January 2016, 7 p.m.) is illustrated in Figure 2; a larger penetration of renewables shifts the supply curve to the right, thereby reducing the energy prices from P to P*.
The second study focused on the Northern European system (including the Nordic countries, the Baltic countries, Poland, and Germany) and was conducted by the VTT Technical Research Centre of Finland through a combined investment and operation modeling approach linking the Balmorel and WILMAR-JMM models. Specifically, a sensitivity analysis on the share of variable renewable energy (VRE) resources was performed, while the portfolio of conventional generation technologies was optimized considering two different time horizons (2030 and 2050). Figure 3 presents the results of this analysis, where (a) and (b) correspond to a 40% and 60% share of VRE, respectively, while both include the current VRE share in the region (22%) for reference. The merit-order effect is evident in both graphs: increasing the VRE share from the current level to 40% and (especially) to 60% reduces the energy prices substantially. Interestingly enough, the prices in 2050 are higher than in 2030, especially in the 60% VRE share case; this is due to the fact that a large part of the existing baseload thermal generation capacity, although remaining in the system until 2030, is expected to retire before 2050.   In this context, maintaining the current market design, which focuses on the trading arrangements for energy as a basic commodity, risks creating a scenario in which the large generation and storage players are unable to recover their investment costs and thus are motivated to leave the market. This critical market challenge is usually referred to as the revenue insufficiency or missing money problem and entails the dangers of compromising the security of supply (considering the potential market exit of conventional generators) and/or compromising the carbon reduction targets (considering the potential exit of low-carbon generators).
In the case of conventional generators, recent European market design initiatives have contributed to addressing the missing money problem. First, the design of balancing markets is continuously refined through the introduction of additional balancing products, the harmonization of procurement and activation processes among different countries, and the gradual shift toward a joint energy and reserve market clearing process. These policy changes are expected to enhance the cost reflectivity of balancing markets and increase the associated revenues of balancing providers. Second, following the U.S. paradigm, some European countries (e.g., the United Kingdom, Ireland, Italy, and Poland) have started implementing capacity markets, remunerating participants that can contribute to the required adequacy levels in a costefficient fashion through competitive auctions; however, capacity remuneration mechanisms remain controversial and have been characterized as market distortive measures.
Finally, the concept of scarcity pricing has been recently highlighted as a means to resolve the missing money problem: during periods of high demand and scarce supply, the energy price is set at the marginal benefit of the demand side, which is often estimated as the value of lost load. Considering the very high value of this marginal benefit, the activation of scarcity prices during a limited number of periods per year can theoretically secure sufficient revenues for generators to recover their investment costs. In this context, the EC has recently recognized [in Regulation (EU) 2019/943] scarcity pricing as a key feature of the future low-carbon electricity market, with Belgium being the first European country that has decided to implement such a mechanism, which is scheduled to start in late 2021.

Market Participation of Renewable Generation
Not only conventional generators but also renewables face significant challenges in the emerging market environment. First of all, most European countries, with the support of the EC policy framework, are gradually abandoning out-of-market incentive mechanisms (such as feed-in tariffs, green certificates, and long-term contracts for differences) that had been introduced in the 1990s to provide the initial push for investments in renewable generation, on the grounds of fully integrating renewables in the deregulated market environment. Second, given the long gate closure times applied in many European markets, the renewable generators' bids are typically based on 12-36 h ahead forecasts in the day-ahead market, entailing significant forecast errors due to the stochastic nature of renewables' output. As a result, the deviations between forecasted and actual output need to be compensated in intraday and balancing markets, with the latter involving the payment of substantial penalties, which compromise the renewables' market profitability. Finally, variable renewable generation is not generally qualified for participation in capacity markets, considering its inherent inability to provide firm power.
Nevertheless, various measures have been recently proposed to address these challenges and enhance the profitability of renewable generators, including both renewables' operational strategies (e.g., advanced forecasting techniques, aggregation strategies) and new market designs (e.g., postponing gate closure times, shortening market resolution, and allowing participation of renewables in balancing markets).
In an effort to quantitatively analyze the effects of such strategies, within the European research program IRPWIND, the normalized value of wind generation (calculated as the difference between its overall market revenue minus its imbalance penalties, divided by the overall energy production) in the Iberian market has been quantified through the agentbased simulation tool MATREM for the following set of scenarios ( Figure 4): ✔ Scenario A: In this reference scenario (with respect to which percentage of wind generation value increase is calculated in the remaining scenarios), the wind generators' market bids are based on deterministic (expected) wind power forecasts. ✔ Scenario B: The wind generators employ a more advanced, probabilistic quantile-based forecast approach. ✔ Scenario C: Multiple wind generators within a given control area are aggregated and then participate in the market as a single entity (with a certain degree of power controllability) to limit the overall forecast errors. ✔ Scenario D: The gate closure time of the day-ahead market is postponed by 2 h (from 12 p.m. to 2 p.m. Central European Time) to take advantage of more accurate forecasts. ✔ Scenario E: Wind generators are allowed to participate in balancing markets, in line with the current market arrangements in certain European countries (e.g., Spain, Germany, Denmark, and the United Kingdom). ✔ Scenario F: Beyond the existing markets, wind generators participate in two new balancing markets-the renewable power band market and the energy reserve market-that have been proposed by the IRPWIND program. These markets are similar to the secondary and tertiary reserve markets, respectively, with the difference that their temporal resolution is 15 min (instead of 1 h), and the wind generators can submit bids up to 15 min ahead of real time to enable them to reduce their imbalance payments. ✔ Scenario G: This scenario is a combination of scenarios B and F, with wind generators participating in the two new markets and employing a probabilistic forecast approach. ✔ Scenario H: This scenario is a combination of scenarios C and F, with wind generators being aggregated and participating in the two new markets. ✔ Scenario I: Beyond participating in the two new markets and employing a probabilistic forecast approach, wind generators can participate in the formation of a new type of bilateral contracts, short-term energy contracts. Following the logic of the new markets, these short-term energy contracts are formed in a 15-min temporal resolution and enable wind generators to trade their energy imbalances.

Prioritization of Renewable Generation in Merit-Order Dispatch
Another important market design issue around renewable generation lies in its prioritization in the merit-order dispatch, with most European markets accepting its curtailment only when the technical limits of the system are breached, on the grounds that such curtailment always increases the operating costs and the CO 2 emissions of the system. However, this assumption is not always valid, as demonstrated by the following example. This example involves a wind generator (which is assumed to be able to produce 100 MWh across the considered 4-h operating horizon), a biomass unit, and a conventional natural gas unit (Table 1), which need to supply a total demand of 160 MW at hours t = 1 and t = 2 and 380 MW at hours t = 3 and t = 4. When the wind generator is prioritized in the dispatch and is forced to deliver its maximum possible output at all hours, the resulting optimal generation dispatch, system operating costs, and system CO 2 emissions are as presented in Table 2. Although the biomass generator constitutes the cheapest available conventional unit and has the capacity to cover the remaining demand at all hours, its maximum ramp rate limit does not allow it to cover the demand at t = 3 (since demand increases from 160 MW at t = 2 to 380 MW at t = 3, while the maximum ramp rate of the biomass generator is 120 MW/h). Therefore, the more expensive and polluting gas generator needs to be activated at t = 3 to cover the remaining 100 MW of the demand.
On the other hand, when the dispatch prioritization of the wind generator is relaxed, the resulting optimal generation dispatch, system operating costs, and system CO 2 emissions are as presented in Table 3. Although the wind generation exhibits lower operating costs and zero CO 2 emissions, its whole output is curtailed at t = 2, enabling the biomass generator to reach a higher output at this hour; the wind generator, along with the biomass generator, subsequently provides the required ramping flexibility at t = 3. As a result, there is no need to activate the more expensive and pollutive gas generator at t = 3, and thus, although the total wind output is reduced by 25% with respect to the scenario with wind dispatch prioritization, the total operating costs and CO 2 emissions are reduced by 3.2% and 100%, respectively. This simple example has demonstrated that the strict prioritization of renewable generation in the merit-order dispatch is not always the most effective strategy in terms of both operating costs and CO 2 emissions. Although this particular example is driven by the ramping requirements of electricity systems, a recent study conducted by the Netherlands Organisation for Applied Scientific Research has presented numerous examples where renewable generation flexibility constitutes an effective market strategy in reducing both operating costs and CO 2 emissions. By adopting similar smart curtailment strategies, renewable generation can be transformed from the cause of flexibility problems to part of the solution (such as contributing to ramping requirements in the above example), thus lowering the system flexibility dependency on conventional generation.

Carbon Pricing
Another crucial policy instrument toward incorporating the ambitious emissions reduction targets within the deregulated market environment is the introduction of carbon markets, which effectively penalize the production of emissions and incentivize investment in low-carbon technologies. In Europe, such a market mechanism, the EU Emissions Trading System (EU ETS), was established in 2005 and remains the EU's flagship policy toward a market-based reduction of emissions. The EU ETS is based on cap and trade principles, meaning that a maximum (cap) is set on the total amount of emissions that can be produced by the system (which is reduced over time to gradually achieve the carbon reduction targets), and a certain number of EU emissions allowances covering this cap are then auctioned and can subsequently be traded. Participants emitting greenhouse gases need to purchase sufficient allowances, lest they face significant fines. In electricity markets, given that the carbon allowance price is passed on by fossil-fueled generators in the electricity price, the revenues of low-carbon generators are increased, partially addressing their missing money problem.
The effectiveness of the EU ETS has been demonstrated in practice, with the EU estimating that the emissions from sectors covered by the system have been reduced by 21% in 2020 with respect to the 2005 levels. However, certain questions have arisen around the long-term economic efficiency of this mechanism, particularly regarding the variability of the CO 2 allowances price. Although the gradual reduction of the CO 2 cap should theoretically lead to an increasing CO 2 price over time, in practice, this price has been unstable. After the global financial crisis of 2007-2008, the CO 2 price dropped from around 25 €/ton to as low as 5 €/ton in 2013; after many years, the price exceeded the 20 €/ton level in 2018, but if the current COVID-19 crisis causes a sustained reduction of energy demand, the price may decline again.
This CO 2 price variability creates significant uncertainties and risks for both potential investors in low-carbon technologies as well as electricity consumers. The potential of a very low CO 2 price discourages investments in low-carbon generation, while the potential of a very high CO 2 price implies an undesired increase in the consumers' energy bills and their subsequent resistance to emissions reduction policies. Although a market stability reserve has been recently introduced to address this challenge by adjusting the number of auctioned allowances, its effect on CO 2 prices is indirect and thus uncertain.
In this context, new designs for reducing the price risks of the EU ETS have been lately brought forward, including the introduction of CO 2 price floors and price ceilings (i.e., minimum and maximum CO 2 price limits). A price floor has already been implemented in the United Kingdom and has been announced in The Netherlands. In an effort to analyze the impacts of these CO 2 market designs, Delft University of Technology has conducted a study through the agentbased model EMLab that simulates self-interested companies' generation investment decisions in alternative technologies (e.g., coal, gas, nuclear, carbon capture and storage, and renewables). Figure 5 presents key results of this study, including the (a) emerging CO 2 prices and (b) CO 2 emissions in Europe in different years (x-axis) under alternative CO 2 market designs (Original ETS, Price Floor, and Price Floor and Ceiling); these results include median CO 2 prices and emissions as well as 50%/90% envelopes as Monte Carlo simulations have been carried out to capture the uncertainties around the evolution of demand levels and fuel prices.
Under all market designs, the CO 2 price is relatively low, and the CO 2 emissions are relatively high during the early years due to the higher CO 2 cap. After about 10 years, however, the CO 2 cap becomes stricter, and thus CO 2 prices increase significantly, reaching very high values in scenarios with high demand growth. Consequently, with a delay corresponding to investment lead times, investments in lowcarbon technologies start emerging, and CO 2 emissions start dropping. After two investment cycles, the market stabilizes, and emissions decline steadily.
Under the current market design with neither price floors nor ceilings (Original ETS in Figure 5), the CO 2 price variability is immense, particularly with respect to the extremely high prices (reaching an extreme value of 500 €/ton) observed after the first 10 years in scenarios with high demand growth. Under a market design with a price floor, the CO 2 price variability is drastically reduced in terms of avoiding both the very low (nearly zero) levels as well as the very high levels observed under the Original ETS design. As a result, the risks associated with low-carbon investments are reduced, such investments emerge sooner, and CO 2 emissions drop faster. Consequently, when the CO 2 cap becomes stricter, part of the required investments has already taken place, and the CO 2 price remains at lower levels.
Finally, under a market design with a carefully considered price ceiling, very high CO 2 prices and subsequently very high consumer energy bills are avoided while the CO 2 emissions reduction targets are not compromised. In conclusion, the introduction of a price floor and a price ceiling in the EU ETS constitutes an effective way to achieve carbon reduction targets with reduced risks for both low-carbon investments and electricity consumers.

The Role and Value of Flexibility
The new flexibility resources, predominantly energy storage and demand-side response (DSR), play a key role in reducing the costs associated with the transition to a low-carbon energy future. An important part of these flexibility resources corresponds to the large-scale technical solutions, such as bulk, long-duration energy storage, that can deal with extreme events of long periods of low wind and solar generation output and DSR from large industrial/commercial consumers that can flexibly schedule some of their processes. However, in the emerging decentralized and digitalized energy paradigm, another very promising part corresponds to small-scale and distributed forms of flexibility sources at the local distribution level, such as residential smart appliances, smart-charging electric vehicles (EVs) with potential vehicle-to-grid capabilities, distributed generation, and distributed energy storage, including heat storage. These resources are owned by small electricity customers who, enabled by the advancements in digital technologies, are gradually transformed from passive electricity consumers to active prosumers, considering their dual ability to flexibly manage their electricity demand and produce electricity through microgeneration. This paradigm change is reflected in the Clean Energy for All Europeans package recently presented by the EC, which highlights the empowerment of energy end users through the active involvement in energy system operation and planning.
According to a comprehensive study conducted by Imperial College London through an advanced whole electricity system model, the potential cost savings brought by the intelligent coordination of flexibility in the Great Britain system are around £3.8 billion/year in a system meeting the Great Britain benchmark emissions target of 100 gCO 2 /kWh in 2030 and around £8 billion/year in a system meeting a more ambitious target of 50 gCO 2 /kWh, as illustrated in Figure 6. The components of these cost savings include.
✔ savings in operating expenses, by the avoided curtailment of zero-cost renewable generation and the more cost-efficient provision of the required balancing services (operating expenditures) ✔ savings in capital expenses associated with reinforcing distribution [D capital expenditures (CAPEX)] transmission (T CAPEX), and interconnection assets (I CAPEX) driven by reduced peak demand levels and the cost-effective management of network constraints ✔ savings in capital expenses associated with investments in conventional generation [G CAPEX (conventional)], driven by reduced peak demand levels and reduced requirements for generation flexibility ✔ savings in capital expenses associated with investments in low-carbon generation [G CAPEX (lowcarbon)] while meeting the carbon target, which is the most dominant benefit in the lower carbon emission scenario of 50 gCO 2 /kWh due to the high cost of firm low-carbon generation technologies (i.e., carbon capture and storage and nuclear), driven by the much more efficient utilization of lower-cost variable renewable generation.  The current European market design does not capture the whole spectrum and full extent of the benefits of flexibility resources, thus hindering their further development. In thve energy market segment, most small consumers and prosumers are still facing flat retail tariffs that do not reflect the time-variable value of energy in the system; therefore, they are prevented from activating their flexibility resources to consume energy during periods of abundant renewable generation and/or produce energy during periods of low availability of renewables. Figure 7 presents the results of a study conducted by Imperial College London, which aimed at quantifying the impacts of domestic demand flexibility (in terms of smart-charging EVs, electric heating with heat storage, and smart wet appliances) on both system operation and the domestic consumers' energy bills in the Great Britain system. Different scenarios have been examined with respect to the percentage of consumers owning the flexibility resources (0, 25, 50, 75, and 100%) and the generation mix (including the current mix in 2020 and the projected mix in 2030). It has been assumed that activation of demand flexibility respects the consumers' service requirements in terms of traveling (for EVs), the indoor temperature (for heating), and the timely completion of the wet appliances' cycles, implying that such demand flexibility does not reduce their overall energy consumption but merely redistributes it in time. Figure 7(a) demonstrates that demand flexibility can greatly reduce the levels of renewable generation curtailment, especially in the 2030 system with higher renewable integration. Figure 7(b) quantifies the energy bill savings that flexible consumers enjoy with respect to inflexible consumers for the same amount of energy consumed, assuming that these bills are based on fully cost-reflective tariffs capturing the system operation conditions. It can be observed that these savings are very high (especially in the 2030 system, where they even reach a level of around 44%), implying that under a cost-reflective energy pricing framework, as we move toward a lower-carbon system, the implications of the temporal patterns of consumers' demand on their bills become more important than their overall energy consumption. It is also noticed that the savings achieved by flexible consumers are reduced as the percentage of these consumers increases, implying that early adopters of flexibility will enjoy the highest benefits.
Second, the majority of European balancing and capacity markets impose excessively strict limits on the type, minimum size, and minimum temporal availability of the participants. In combination with the lack of regulatory clarity around the role of aggregators in many European countries, the value of distributed flexibility in reducing the system balancing and capacity costs remains largely unexploited. Notable examples include forbidding demand-side resources from accessing certain markets or not participating on a level playing field with large-scale generation (e.g., shorter contract lengths). Furthermore, European balancing and ancillary services markets generally ignore the time-coupling operating properties of DSR since each ancillary service product is cleared independently. As a result, market outcomes are not fully cost reflective and may overestimate the value of some flexible resources. As an example, a study conducted by Imperial College London has quantified the value of frequencyresponse service provided by thermostatically controlled loads in the Great Britain system under the independent and simultaneous clearing of frequency response and reserve services. In this example, a case when refrigeration provides the primary frequency control by reducing its consumption will be naturally followed by a load recovery effect (i.e., the demand in a subsequent period will be higher than the level it would follow if the provision of frequency response had not taken place to restore temperature at the desired setpoint), implying that the secondary reserve requirements of the system may increase. Therefore, the actual value of the frequency regulation service when accounting for this effect is visibly lower than the one projected by the current independent clearing approach.
Moreover, the location-specific component of the distribution network charges constitutes a very small proportion of the overall charges in most European countries, and the largest amount of network costs is socialized, preventing distributed flexibility resources from taking actions to avoid/defer distribution network reinforcements. Last but not least, the value of flexibility resources in reducing the low-carbon generation investments required for the achievement of carbon targets (which constitutes the most significant value stream in the low-carbon future, as illustrated in Figure 6) is not currently captured by any European market design, to the best of the authors' knowledge, and constitutes a key market design challenge going forward.

Geographical Integration of Electricity Markets: The European-Wide Approach and Local Energy Markets
As previously discussed, a cost-effective transition to the low-carbon energy future involves a combination of largescale renewable generation and the deployment of smallscale distributed flexibility resources at the local level. In this context, another major policy challenge lies in the introduction of suitable market mechanisms at multiple geo-graphical levels, ranging from the European-wide level to the local community level.
Concerning the former, previous work has demonstrated that a coordinated European-wide approach for the integration of renewable generation can offer very significant benefits compared to a member state-centric approach by taking advantage of the significant geographical diversity of renewable energy resources' availability, including the higher capacity factors of wind generation in Northern Europe and the higher capacity factors of solar generation in Southern Europe. Specifically, if such diversity is combined with a full harmonization and integration of the different countries' electricity markets, the same amount of renewable energy can be produced with 150 GW fewer renewable generation capacity with respect to the member state-centric approach, entailing around €200 billion of savings in capital investments until 2030. Although such a European-wide approach has been outlined in the European Renewable Energy Directive, it has not yet been realized. Furthermore, interconnections to the Middle East and Africa could potentially further increase these benefits by exploiting the high solar generation availability in those regions.
At the other end, despite the massive value of distributed flexibility resources enabled by the digitalized energy paradigm, the effective integration of large numbers of such small resources in electricity markets is extremely challenging due to scalability limitations and privacy concerns raised by the end consumers/prosumers. In this context, local energy markets (LEMs) constitute a new market mechanism attracting continuously increasing interest. LEMs enable the direct trading of energy and flexibility among the end users of a local community, coordinated either in a centralized fashion (e.g., by an independent community manager) or in a fully distributed fashion through emerging peer-to-peer trading architectures.
Beyond addressing scalability and privacy concerns, a LEM promises a number of significant benefits, including a) limiting the energy dependency of active consumers/ prosumers on the incumbent electricity retailers and consequently enhancing the competitiveness of the latter; b) avoiding distribution network reinforcements as a result of matching local demand with local generation; c) enhancing the engagement of local end users in system operation by creating a local identity and promoting social cooperation; and d) revitalizing the local economy by shaping opportunities for local investment, creating new jobs at the community level, and promoting self-sufficiency. The EC has recognized these The value of flexibility resources in reducing the low-carbon generation investments required for the achievement of carbon targets is not currently captured by any European market design.
benefits by establishing and promoting the concept of local energy communities.

The TradeRES Vision
The vision of the recently initiated Horizon 2020 TradeRES project (www.tradeRES.eu) lies in developing and testing innovative electricity market designs that will enable the cost-effective and secure development of a nearly 100% renewable power system and realize the full extent of the system-wide benefits of flexibility resources. Such market designs should be capable of addressing the key challenges identified in this article, including a) the missing money problem faced by both renewable and conventional generators, b) the participation of renewable generation in ancillary services markets and the provision of flexibility, c) the design of effective carbon emissions markets, d) the incorporation of small-scale distributed flexibility in energy and balancing markets, and e) the pan-European harmonization of electricity markets and the economic utilization of cross-border interconnections. Following the previous discussion, one of the main objectives of the project involves the development of an integrated European market architecture that encapsulates the pan-European market, national/regional markets, and local energy communities in a fashion that enables the maximum utilization of available renewable generation and flexibility resources, as reflected in Figure 8.