A Framework for Enabling Cloud Services to Leverage Energy Data

Cloud services have been well integrated into various sectors of contemporary societies such as transportation, healthcare, and work. However, the energy sector has made little progress in offering cloud services to energy consumers. In some cases, even basic features (such as energy consumption monitoring and visualization of households) are still unattainable. To foster innovation in cloud-based energy services, governments implement regulations that moderate access to energy data via the energy providers’ platforms. Despite these regulations, cloud service providers may still struggle to aggregate energy data because each energy provider may be using different procedures, authentication mechanisms, and data semantics. Consequently, aggregating energy data at scale becomes challenging, as it requires achieving compatibility with diverse platforms. To address this challenge, we propose a cloud-based framework that handles all interactions with energy providers and prepares the data for further processing by services. Additionally, we analyze the socioeconomic impact of our approach, and we outline novel use cases that may emerge due to the proposed framework.


I. INTRODUCTION
Many sectors of modern societies (such as transportation, public administration and healthcare) have been improved significantly by the use of cloud services.However, the use of cloud services in the energy sector is still limited.The prime reason is that access to energy data is usually restricted to the energy provider.Actually, in many instances, even energy consumers cannot access detailed information regarding their energy consumption.This lack of transparency has become an obstacle in mining energy data and exploiting its potential for creating novel cloud services [1].To encourage the development of cloud services in the energy sector, governments create regulations that moderate access to energy-related data.For instance, the European Directive (EU) 2019/944 establishes the right to access energy-related production and consumption data for energy consumers and entities of their choice [2].Similar directives in the United States of America also spread across the states, such as Decision (D.) 14-05-016 in California, and the Administrative Rules R 460.101 to R 460.169 in Michigan, among others [3].These regulations aim at enabling data-based cloud services while also allowing energy consumers to share their data in a privacy-preserving manner.
Energy providers strive to build platforms that enable the sharing of energy data based on local regulations and best practices [4].However, since the regulations and practices can vary (especially between different regions), the resulting platforms become diverse, which makes it impractical to aggregate data from different regions.For example, each energy provider might be using different procedures, authentication mechanisms, semantics, and data formats, among others.This hinders the mining of energy data [5].
To enable access to energy data for service providers while also abiding by local regulations, in this paper, we propose a novel framework.Our framework includes mechanisms to manage the consumer's consent for sharing data (to avoid privacy infringement), and to aggregate energy data from multiple consumers regardless of their energy provider and the potentially different local regulations.Moreover, the proposed framework consolidates the data into a uniform format that can be easily processed by cloud services.In summary, our contributions include: i) The motivation of this approach based on a survey of participants who try to access their energy data.
ii) The detailed description of the proposed framework.iii) An overview of applicable use cases.
The remainder of this paper is organized as follows: Section II discusses the motivation behind our work, and Section III presents the proposed approach.We then explore potential use cases and their impact in Section IV, and we outline related work in Section V. Finally, Section VI concludes this paper and provides future research directions on this topic.

II. MOTIVATION
To confirm that cloud services in the energy sector are indeed limited in functionality, we conducted a survey at the University of Vienna (in Austria) during the winter semester of 2022.In this survey, 32 students were instructed to download their energy consumption data and report on their experiences.The results of the survey are outlined in Table I with columns that follow this structure: 1) The name of the country.
2) The number of attempts to download data in this country.
3) The examined energy providers for each country.

4)
The region of each energy provider.5) The services that each energy provider offers.Various obstacles to downloading data were reported, such as: i) Not knowing who is the responsible contact, e.g., the grid operator, or an energy retailer.ii) Some smart meters in households were not yet connected to accounts.iii) Not knowing what credentials must be used for accessing the providers' online platforms (e.g., cross-nation credentials, or creation of new accounts.iv) Some students mentioned that while it was not possible to download the data, the energy provider posted the data via mail in a printed letter.Regarding available cloud services, most students reported that a website was available showing information about the active subscription with the energy provider.In some cases, this website offered additional services, as shown in Table I.
In conclusion, we note that despite the diligent efforts of policymakers to make the energy data available and enable innovative cloud services, energy services are, in general, neither highly available nor innovative.Few participants managed to visualize their consumption, and only one was able to use more advanced services (in Norway), while most did not even manage to download their data.Based on the results of this survey, we identify two crucial gaps in the way of developing energy cloud services.First, an energy provider may have no significant benefit from developing and offering services.The reason is that such services can only be offered to existing customers, thereby missing the goal of addressing a larger audience, and potentially attracting new customers.Second, there is currently no competition in the market for developing energy services because the energy data is only available to energy providers.Based on these gaps, we define the following RQs which drive the design of the proposed framework in the next section: 1) How to enable an energy provider to offer its services to a larger audience than existing customers?2) How to allow other organizations than energy providers to enter the market of developing energy services?

III. FRAMEWORK FOR ENABLING ENERGY SERVICES A. Requirements
Various requirements need to be met by a framework that enables cloud services, such as security and availability [6].In addition, the proposed framework shall satisfy Requirements 1-4 which are listed below and apply specifically to the energy sector.
1) Consent-based: Every energy consumer has to be able to decide who can process their data for avoiding privacy invasion.Therefore, explicit consent from a consumer needs to be provided to the framework [7].2) Decentralized: The framework has to be decentralized so that every cloud service provider can configure it individually to operate only with energy data from consumers who have agreed.This is very important because access to energy data may be regulated by relevant laws.Thus, there shall not be a central component that aggregates all the data [8].3) Scalable: The framework has to be scalable because a cloud service provider may have multiple services running on data from many energy consumers [9].4) Extensible: The framework needs to allow extensions that improve its compatibility with energy providers.This is important to achieve large coverage of regions.

B. System Model
To make the proposed approach comprehensible, we present a high-level view of a system that is based on our framework and showcases the intended functionality.This system, which is depicted in Fig. 1, includes three types of actors: 1) Energy consumer: A legal person responsible for the energy consumption at a site, e.g., a household or a factory, who is also the owner of the produced energy data.The energy consumer is also referred to as consumer.2) Eligible party: This is an entity (e.g., a cloud service provider, a grid operator, or a research group) that has the consent of consumers to process their energy data using cloud services.3) Energy provider: This is the entity that handles the process of supplying electricity to the site of a consumer.
The energy provider can be the company that operates the power grid, an electricity retailer, or a metered data administrator [10].This provider stores the data from smart meters, e.g., for billing purposes.In the system of Fig. 1, a consumer (e.g., in a household) can browse the marketplace for interesting cloud services that rely on energy data.The marketplace is where eligible parties can advertise their services.These are services that process energy data from one or more sites and produce useful results.An example can be a service that provides interactive visualizations of energy consumption for a predefined period.When a desired service is found, the consumer acquires from the marketplace the web address of the eligible party that offers this service.The consumer then contacts the eligible party directly using this web address to request the service.
A service offered by an eligible party may require historical data, i.e., data about the energy consumption of the past, and/or real-time data, i.e., data about the current energy consumption.The former can be acquired from the energy provider as long as the consumer has given their consent.Importantly, realtime data is typically not available from energy providers.The reason is that energy providers preprocess the collected data, e.g, to fix noisy, lossy, and out-of-order data, before it is stored and used for other purposes, such as billing [11].To avoid the delay of this preprocessing, in our system, real-time data is sent from the site (e.g., the household) to the eligible party over the Internet.Interestingly, in the system of Fig. 1, a consumer can use cloud services from one or more eligible parties, e.g., household 1 uses services from all three eligible parties, while household 2 uses only one service from eligible party 3.In addition, each service from an eligible party may operate on data from one or more households.For example, the service from eligible party 2 operates on data from household 1, while the service from eligible party 3 operates on data from households 1 and 2.
Fig. 2 shows a sequence diagram of the basic interactions among an eligible party, a household, an energy provider and the marketplace.Initially, the eligible party advertises a cloud service in the marketplace (Step 1 in Fig. 2).The consumer can then search the marketplace for services based on custom criteria, e.g., visualizations, analytics or predictions (Step 2).The response of the search includes identifiers for the services that match the criteria, along with identifiers for contacting the corresponding eligible parties, e.g., web addresses (Step 3).Then, the consumer contacts the eligible party and requests to use a service (Step 4).Since the real-time data needs to be sent from the household, the eligible party dynamically creates an interface for receiving this data and sends the address of this interface to the consumer (Step 5).The consumer then configures an in-house device (Step 6) to start streaming realtime data to the eligible party using the interface (Step 7).Subsequently, the consumer requests from the energy provider that the eligible party accesses the historical data (Step 8), and the energy provider responds with an access token (Step 9).The token is then sent to the eligible party (Step 10) that uses this token for downloading the historical data from the energy provider's platform (Steps 11 and 12).Notably, the process of providing consent to the eligible party for accessing historical data may vary.Nevertheless, we consider the aforementioned to be a representative example.

C. Internal Structure of the Proposed Framework
For a deeper understanding of our approach, Fig. 3 shows the internal structure of the proposed framework in a simplified scenario with one eligible party, one energy provider, and one household.Our framework includes components of the eligible party (apart from the services), the household, and the marketplace (i.e., yellow, orange and green blocks in Fig. 3).These components are: User Interface: This is the website of the eligible party and allows the consumer to select services, and to provide their consent for access to historical data.
Consent Facade: This component manages and stores the consent of consumers, and helps with meeting Requirement No. 1 of Section III-A.Notably, since the process of giving consent may vary among energy providers, different procedures shall be supported (as mentioned by Requirement No. 4 of Section III-A).
Admin Console: In addition to having the consent of the consumer, eligible parties may need to register with each energy provider and receive a unique eligible party ID.The  Admin Console provides an interface for the eligible party to manage consents and eligible party IDs.
Interoperable Communication: This component is responsible for data interoperability which refers to receiving energy data in various forms (e.g., different syntax, structure, and semantics), translating to a uniform format, and storing it.Interoperability can be achieved using adaptors, i.e., one adaptor for each energy provider handling data translation.This component aids in meeting Requirement No. 4 of Section III-A.
Storage: This component stores the state of the framework which may include the energy data, and various information regarding credentials, consents, and configurations.Integrating one Storage component for each instance of the framework ensures that the data is not aggregated through a central component.This helps with satisfying Requirement No. 2 of Section III-A.
App: Every eligible party can have various apps which execute the logic of the services.A service may consist of one or more apps that operate sequentially or in parallel [12].The results of the services become available to the consumer through the User Interface.
The components of the Energy Provider are the Data Portal and the Admin Portal [13], as shown in Fig. 3.The Data Portal provides access to historical data for either the consumer or an eligible party that has received the necessary consent.The Admin Portal offers an interface for consumers to provide their consent to eligible parties, e.g., by generating access tokens.This component also allows eligible parties to register so that they can later request access to historical data (as shown in Step 11 of Fig. 2).
Every household has one component which is the Administrative Interface for In-house Data Access (AIIDA).AIIDA is deployed at the edge of the network [14], [15].Specifically, this component runs on a device, e.g., on a Raspberry Pi computer, that is connected to the household's smart meter, e.g., via a P1 port [16].The smart meter is a device installed in every household by the energy provider, that reads the energy consumption values in real time.AIIDA has access to the Internet through the household's LAN (Local Area Network).This is important because the consumer needs to access the configuration of this device via LAN (e.g., through the localhost) to add the eligible party's interface, as discussed in Section III-B.After this process, AIIDA reads real-time data from the smart meter and sends it to the eligible party via the Internet.In addition to the smart meter, AIIDA can be connected to other energy metering systems such as dedicated home automation devices, electrical vehicle chargers and photovoltaics [17].This way, the data from such systems can also be sent to the eligible party for processing.
Finally, the marketplace, which can be viewed as a central point for consumers to search for cloud services, is a federated service that consists of a collection of multiple marketplace instances which communicate with each other, e.g., in a peerto-peer manner [18].Some eligible parties may want to run their own marketplace, e.g., due to privacy concerns, while others might prefer to run a marketplace together.For this reason, we consider the marketplace as a federation that includes multiple marketplace instances of both cases.
Notably, the proposed system structure allows an eligible party to leverage energy data from many sites without using any centralized components.This ensures that the energy data is not transferred to any entity that does not have the consent of the consumers, and aligns with Requirement No. 2 of Section III-A.Requirement No. 3 can be met by the communication mechanisms of the framework.While it can be challenging to handle many data streams at the same time, existing approaches in the field of publish-subscribe communication can aid in satisfying this requirement.For example, message brokers and message queues such as Apache Kafka or MQTT are designed for scenarios with multiple data providers and data consumers [19].

IV. USE CASES
Various research approaches focus on use cases that depend on cloud services which process energy data, such as energy consumption recommendation systems, in-house optimizations, and off-peak pricing, among others [20].In addition, many novel use cases focus on integrating modern energy sources into the power grid, such as the batteries of Electric Vehicles (EVs) and photovoltaic systems.Such use cases, which are also depicted in Fig. 4, can benefit from our approach because the proposed framework can be used for collecting the required input energy data.In addition, our framework can facilitate the exchange of information among different services, leading to synergies between different use cases.
The proposed approach can also be used to address RQ 1 presented in Section II.Currently, energy providers are able to offer certain cloud services to their consumers, as discussed in Section II.However, these services are often limited to the consumers of each energy provider.This is depicted in Fig. 5(a  because this is their energy provider.Thus, there is a vendor lock-in that affects the services that are available to each consumer.Fig. 5(b) illustrates the integration of our framework by Energy Provider 1, i.e., this provider also takes the role of the eligible party.Same as in Fig. 5(a), the consumers in both regions can still use the services of their respective energy providers.However, the consumers of R 2 can now also use the services of Energy Provider 1, on the condition that they give their consent.This is enabled by our framework which collects real-time data from AIIDA instances, and historical data from Energy Provider 2, as depicted by the green-colored arrows in Fig. 5(b).As a result, the services of Energy Provider 1 can mine data from R 2 .
This example showcases the use case of enabling the cloud services of an energy provider to cater to a broader market.Interestingly, this use case can have a huge impact on the energy services market because the consumers are no longer bound by the vendor lock-in, i.e., the consumers of R 2 are free to select services by the energy provider of their choice.Thus, in an environment where energy providers use the proposed framework, conditions of competition among services are formed.Consequently, as the theory of competitive markets suggests, an impetus for growth in the cloud services market can be anticipated, which can give rise to a variety of new applications and features, as well as greater freedom of choice for the consumers.Moreover, competition can lead to a reduction of information asymmetry and information costs, which promotes the cloud service provider's efficiency and sustainability [21].
Furthermore, the proposed framework can help with addressing RQ 2 (discussed in Section II) regarding allowing other organizations than energy providers to enter the cloud services market for the energy sector.Interestingly, in the same way Energy Provider 1 takes the role of the eligible party by integrating the framework in Fig. 5(b), other organizations (e.g., artificial intelligence companies, or research groups) can integrate it too.The only requirement for mining energy data using our framework is that the eligible party has the consent of the consumers.Thus, even though the energy services market is currently controlled primarily by energy providers, our approach enables any organization to take the role of the eligible party.This allows for even more competition and further enhances the benefits discussed above in the context of RQ 1.As a result, the consumers are now not limited to their energy provider or any energy provider but are free to choose any cloud service provider.Some additional use cases that can benefit from our framework to optimize power consumption and promote sustainable living are outlined in the following.

A. Smart Prosumer Flexibility
To reduce the carbon footprint, modern societies have started a transition towards higher renewable energy usage in the power grid.This transition has led to a shift from traditional passive energy consumers to a new role of active prosumers [22].A prosumer is a typical consumer who also generates electricity, e.g., through rooftop photovoltaics.Prosumers can play important roles in balancing the power grid by adjusting their production/consumption patterns based on the grid's supply and demand.This can help to avoid congestion, especially during peak demand periods [22].To aid in making collective decisions that benefit the power grid and the prosumer's interests, access to data from the grid, other prosumers, and pricing models, is crucial.By following the proposed approach, prosumers can search for recommendation services in the marketplace, agree to give their data to an eligible party and receive notifications about optimized decisions based on services that consider various data.

B. Electric Vehicles -Power Grid Integration
Modern EVs can have high energy storage capacities, e.g., to support reliable performance and advanced onboard computations.The charging of EVs at scale has raised skepticism regarding the imposed load on the power grid, and whether EVs can help the power grid during peak demand [23].Various approaches have been proposed to optimize EV charging such as controlled charging, off-peak charging, and smart charging [23].Such approaches aim at facilitating EV charging in a fashion that prevents adverse impacts while pursuing peak shaving.However, these approaches rely on diverse realworld data (e.g., from multiple EVs) that can be difficult to acquire.By taking the role of the eligible party, EV-charging service providers can focus on developing tailored schedules and recommendations for efficient charging of EVs while aggregating the required data using our framework.

C. Smart Microgrids
Microgrids are small-scale power grids that can operate independently (i.e., stand-alone) or in coordination with the larger power grid (i.e., grid-connected).Microgrids often rely on distributed renewable energy sources, such as solar and wind power, and integrate energy storage systems to provide a reliable power supply for sustainable living.To support microgrids and connect them to the power grid efficiently, a significant amount of information is required, including crossdomain and real-time data [24]- [26].A major challenge in microgrid deployment is that they have to be investigated for each region separately due to the reliance on simulated and region-specific data.Therefore, it becomes difficult to make generally-applicable conclusions for mass deployments [27].Our framework can help with this issue by providing access to a larger pool of real-time and historical data (provided that the data owners agree), including data from smart grids and microgrids.

D. Sector Coupling
The integration of different sectors such as electricity, gas and transportation, holds a promising potential to promote the decarbonization of societies.To achieve such integrations, the European Green Deal has defined the Energy Systems Integration strategy identifying six pillars for overcoming existing barriers [28]: 1) A circular energy system that prioritizes efficiency.2) An increased electrification of the energy system mostly based on renewable energy sources.3) Fostering renewable and low-carbon fuels for hard-to-abate sectors.4) Creating energy markets suitable for decarbonization and distributed generation.5) An integrated energy infrastructure.6) A digitalized energy system that supports innovation.
The last pillar, in particular, can be seen as an enabler for the development of the other pillars.Digitalization can facilitate efficient management of energy systems in terms of operation, planning, and investments through the reduction of transaction and coordination costs [29], [30].The decrease in transaction and coordination costs also translates into lower entry barriers in the energy services market.Low entry barriers and high commercial value can give rise to new business models and novel services that tackle existing concerns regarding security, privacy, trust, and reliability in the energy sector [31].Such concerns can be addressed by fostering innovation using cloud services that are developed by eligible parties using our framework.

E. Power Grid Operation
The coordination of the power supply and demand separately and jointly has been recognized to result in various economic and system benefits for planning and operational purposes [32].These benefits have, until recently, been confined to individual transmission and distribution networks.However, this is no longer the case [32].Data sharing and cloud computing can further extend such benefits to consumers and utilities, as well as create new business models for coordination across the power grid operators, i.e., Distribution System Operators (DSOs) and Transmission System Operators (TSOs).There are also considerable benefits in adding to this coordination data from distributed flexible supply and demand, i.e., from Distribution Network Operators (DNOs) [33].Harvesting the potential benefits of coordination among all the operators across the power grid requires technological, regulatory, and business model innovations.
Our framework can help to achieve this coordination by acting as the link to share data between operators.This may allow tapping into the potential benefits by enabling access to real-time and historical energy data from various sources.Consequently, the control of bi-directional physical flows of power between DSO and TSO can become easier to manage, and virtual utilities that combine distributed generation and demand data may become possible.In addition, better coordination of TSO-DSO linkages can create new opportunities.DNOs and TSOs can benefit from sharing distributed supply and demand data using cloud services across jurisdictions to unlock these opportunities.For example: • Coordination and better use of the TSO-DSO interfaces allow network utilities to economize on costly network reinforcement investments.• Balancing Responsible Parties can extend the reach of their flexibility services to other jurisdictions than their designated ones.• Eligible parties can offer their flexibility and demand response services by combining dispersed resources.For instance, aggregators can coordinate the charging and discharging of EV batteries across DNOs to help manage or bypass transmission congestion [34].
V. RELATED WORK Related work stems from the contributions of papers that propose architectures for energy data collection.Pires et al. [35] propose an architecture for enabling cloud services to use energy data from smart meters in the context of smart cities.In this architecture, smart meters are connected to a custom gateway (e.g., a Raspberry Pi computer) that is configured to send data to services via the Internet.Karimi et al. [36] present an architecture with a Meter Data Concentrator component that aggregates and concatenates data from smart meters to reduce the volume of data that reaches the services.Mustafa et al. [37] propose an architecture with smart meters, data communication companies (i.e., eligible parties that run services), and output parties (i.e., the entities that consume the results of the services).The authors also discuss aspects related to privacy and security.
There are also contributions on how to access energy data from papers that aim at proposing services, e.g., for data analytics and predictions [38].For instance, Iyengar et al. [39] discuss services that process smart meter data, and provide insights related to how the energy consumption of a small city is affected by the weather and the age of the buildings.To conduct this study, the authors rely on a dataset acquired from a utility company.Guo et al. [40] propose a method based on a Hidden Markov Model, that detects an appliance's energy consumption (e.g., from a refrigerator) using aggregated energy data from households.For the experiments, the authors rely on simulated and real data, e.g., from publicly available datasets such as the Reference Energy Disaggregation dataset.Dong et al. [41] present an approach for predicting single-day energy consumption based on a Random Forest model.This work is implemented using a publicly available dataset from the Low Carbon London project.
Finally, there is related work from papers that focus on the access and collection of energy data from households [38].For example, Asghar et al. [42] discuss the collection of smart meter data with emphasis on privacy aspects, and highlight related privacy legislations such as the need to have the consumer's consent.The authors also mention the transmission of data from smart meters to service providers via power-line communication or cellular networks.Wijaya et al. [43] discuss the importance of aggregating smart meter data for demand forecasting using feature selection.The authors present algorithms and experimental evaluations of forecasting energy consumption of many households using data acquired from the Commission for Energy Regulation in Ireland.
Overall, we note that various contributions in the literature focus on the collection of energy data for enabling services.However, there are still crucial aspects that can be further researched.For example, how to acquire the consent of the consumers, how to access historical data, and what is the potential socioeconomic impact of cloud services in the energy sector.These aspects are within the scope of our work.

VI. CONCLUSION
To promote the development of cloud services for energy consumers, in this paper we present a cloud-based framework for aggregating energy data from multiple sites.This framework can be deployed and operated by an eligible party, i.e., an entity that has received the consent of the consumers to process their data.By using our framework, the eligible parties can focus on creating innovative cloud services, without addressing the challenge of achieving compatibility with the different procedures required by the various energy providers.Furthermore, we analyze the potential socioeconomic impact of our approach along with novel use cases that may emerge due to the proposed framework.
Future work on this topic includes designing efficient mechanisms that provide the functionality of the framework's components.In addition, the processing of real-time and historical data from various regions can create new opportunities for energy service providers, e.g., to focus on designing novel services, and for research groups that can use this data for validating novel data analysis techniques for the energy sector.

Fig. 5 :
Fig. 5: (a) Example of two energy providers each one offering services to consumers in a specific region.(b) Integration of our framework by Energy Provider 1 that now offers services to consumers in both regions.

6 )
The format of the downloaded

TABLE I :
Results from a survey of participants in different countries trying to download their energy data.In total, 32 participants made 42 attempts to download data from various energy providers with overall success rate: 7/42.
data (marked with "None" if no data was downloaded).7) The success rate, indicating how many attempts to download energy data were successful.Importantly, most international students attempted to access data from their residences in Austria, and the residences in their home countries.For this reason, the number of attempts (i.e., 42) is bigger than the number of participants (i.e., 32).The dashes ( -) in the table