Securing Smart Metering applications in Untrusted Clouds with the SecureCloud Platform

Data security in smart metering applications is important not only to secure the customer privacy but also to protect the power utility against fraud attempts. Usual deployment of metering applications rely on the power utility infrastructure, assuming its Advanced Metering Infrastructure (AMI) as trustworthy. This paper describes the design and deployment of a smart metering system focusing on the security of the AMI (smart meters, data aggregator on the field, Metering Data Collection system and metering database) considering the data processing on untrusted clouds. We discuss one use case of the SecureCloud project, an ongoing project that investigates how security and privacy requirements of smart grid applications can be met with a secure cloud platform based on Intel SGX enclaves. The paper describes the components of the advanced metering system as well as the security approach adopted to meet its requirements. A smart metering application has been prototyped in the SecureCloud platform and the integration challenges are discussed from the perspectives of security, privacy and scalability.


INTRODUCTION
e increase of Smart Grid applications in energy distribution utilities poses new demands for data processing and storage as well as data privacy and security. e migration from old energy metering systems, based on manual meter reading, to the concept of Advanced Metering Infrastructure (AMI) sets new requirements for communication systems, data storage and near real time processing of Big Data. Smart metering data also have particular requirements of data security and privacy depending on the stakeholder goal Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or a liate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only. W-P2DS' 18 [13]. For example, privacy requirements of applications like billing di er from those of energy demand monitoring and requirements of applications for controlling customer energy consumption di er from those of fraud detection, at the same time all of them use the same metering measure databases.
Assuming cloud computing can assure high scalable data storage and processing, Smart Grid applications can bene t from cloud computing solutions as long as the provided security and privacy meet the requirements of the application. In particular, this paper focuses on applications for energy distribution utilities, mainly for those with a large number of customers, handling large volume of data from applications such as Metering Data Collection (MDC), billing and Metering Data Management (MDM).
AMI is part of the Smart Grid critical infrastructure of an energy distribution utility: from the communication system of a meter in the eld until the metering data storage and processing, strict requirements of privacy and security should be met as discussed by several studies [5,14,22,24] and guidelines [16,23]. In special, for the Brazilian energy metering market, security is a concern due to the high level of fraud a empts (classi ed as non technical losses) in energy metering systems [10]. According to the Brazilian Association of Electrical Energy Distributors, in 2015, 5.75% of total delivered energy (27 TWh) was lost due to energy the and fraud in the metering systems [9,12]. Currently, a acks are mostly focused in metering tampering and in irregular connections but it can migrate to other components, such as communication, concentrators, Metering Data Collection (MDC) and billing systems with the increase in use of AMI.
Nowadays, in the Brazilian energy market, most of the industrial and commercial consumers are using AMI systems. is group represents only 7.7% of total customers but accounts for more than 55% of total energy consumption [12]. Assuming the increase of AMI usage, the adoption of cloud computing applications for utilities can be economically feasible if data security and privacy requirements of Smart Grid applications could be met. e SecureCloud platform aims at providing solutions to critical infrastructures, allowing secure processing of sensitive data within untrusted cloud environments [18]. ese solutions can be designed to meet the requirements of privacy and security of Authentication and MDC applications, as well as secure storage schemes for metering databases, for example. e platform uses a new cryptographic hardware from modern Intel CPUs, named So ware Guard Extensions (SGX) [2,8] to build enclaves (protected memory areas of code execution). SGX aims to provide integrity and con dentiality guarantees to secure sensitive computation even when all the privileged so ware (kernel, hypervisor, etc.) are potentially malicious [8].
One use case of the SecureCloud Project refers to a secure smart metering application migration to untrusted clouds making use of the SecureCloud platform. We are particularly interested on assessing the challenges faced by so ware developers in this migration process.
is paper reports our achievements as follows: Section 2 describes to the SecureCloud infrastructure, its architecture and basic concepts; Section 3 describes the Secure AMI application scenario, its requirements, components and challenges; the details of the deployment are described at Section 5 and the integration to the SecureCloud platform at Section 6. e nal section discusses the deployment details and the approaches used to overcome the challenges faced so far.

SECURECLOUD INFRASTRUCTURE
Secure remote computation refers to executing so ware on a remote computer owned and maintained by an untrusted party, with integrity and con dentiality guarantees. In general, secure remote computation is still an open problem. Figure 1 shows the baseline infrastructure of the SecureCloud Platform. An application consists of a set of micro-services [15] connected via an event bus [18]. e application logic of each micro-service lives within an enclave. e micro-service runtime exists outside of the enclave. ese runtime functions only access encrypted data. Encryption and decryption of this data is performed automatically and transparently within the enclave. is approach limits the amount of code added to the Trusted Code Base (TCB).
To deploy a micro-service, the SecureCloud platform o ers secure containers on top of the untrusted stack of the cloud provider in order to add con dentiality and integrity to Docker containers 1 .
is enables system administrators to build secure container images within a trusted environment and to run them in an untrusted cloud. e creation of secure containers for the SecureCloud Platform is simpli ed by a Secure Linux Container Environment, called SCONE [3] that secures existing applications with SGX.
SCONE provides the micro-service with an interface based on external system calls, which are shielded from a acks. To protect itself from user space a acks, SCONE performs sanity checks and copies all memory-based return values to the inside of the enclave before passing the arguments to the micro-service. SCONE further (i) transparently encrypts and authenticates data that is 1 h ps://www.docker.com/what-docker processed via le descriptors, and (ii) provides acceptable performance by implementing tailored threading and an asynchronous system call interface [18]. SCONE integrates with existing Docker environments, and ensures that secure containers are compatible with standard containers.
SecureCloud also proposes SGX-based secure data communication between containers that protects messages exchanged by encrypting them inside of enclaves via an event bus. Encryption and decryption are automatically performed within enclaves. Decryption keys are stored securely within enclaves only. Authentication is also provided, as well as additional security properties of the bus, such as ensuring freshness of the delivered messages (ensured by the event bus stubs that run inside an enclave) [6]. Micro-services communicate with external clients and services through secure communication channels [4].

SECURE AMI APPLICATION
e smart metering application system consists of a set of smart meters connected to an aggregator, which is responsible for providing a backhaul connection to the MDC and the metering database. We explore the deployment of a Secure AMI application based on untrusted clouds.  In the Secure AMI application, the MDC remotely communicates with the smart meter (SM) through the aggregator (used to bridge the backhaul network into the wireless mesh network [1]), to convert the cryptography scheme to connect to the SMs through its protocol [7], and to insert this information as payload, so it can be sent to the meter. e MDC Application periodically sends requests to the SM in order to collect information stored in the database. Additionally, the aggregator is responsible for the wireless network management.

Security Requirements
Based on the scenario and the use case outlined above the following security requirements can be derived: • Con dentiality (including privacy) -Detailed load signature must be kept con dential.
Only the a ected consumers should be able to see their consumption data in full detail. -Di erent utility's computing systems should be allowed to access the data in di erent aggregated manner, only to reinforce user's privacy (di erent computing systems should have di erent data access privileges). • Integrity (including non-repudiation) e integrity as well as the accountability (including non-repudiation) must be ensured. is is critical due to the billing and load management.
• Availability -Some availability needs to be ensured but there is no warranty for real-time access to the necessary data.

AMI IMPLEMENTATION 4.1 Communication
e meter connects to the aggregator to synchronize, authenticate, and exchange data. e exchanged information is forwarded from the aggregator to the Authentication Application which informs the MDC about the available meters and sets the information needed to ll the phasor report. e phasor report is periodically sent (every 15 minutes) with all meter information required by the energy utility company. e communication between the meter and the MDC follows ABNT (Brazilian Association of Technical Standards) NBR 14522 [11] protocol, which is packed inside a proprietary protocol [7] de ned for electricity metering system. e current implementation uses this proprietary protocol to manage communication between the aggregator and with multiple devices in large networks: a preliminary negotiation between the aggregator and a meter de nes the intent for data requests, and a limited number of "tokens" at a time (less than the number of devices) is used to coordinates the reception of answers from meters. e meter response is validated by the Authentication Application and the requested data is loaded into the database by the MDC Application.

Smart Meter
e smart meters were developed to enhance the security via an embedded Trusted Platform Module (TPM) and also to add advanced features. e embedded TPM implements both symmetric cryptography (AES) and digital signature based on asymmetric algorithms (RSA), following the recommendations of the Guidelines for Smart Grid Cyber Security from NIST [23] and following the PCB (Public Key Infrastructure) for authentication. ey were deployed as single and polyphase energy meters, following the Brazilian standards for energy meters to residential and small commercial consumers, supporting AMI structure. e main characteristics of this smart meter are its use of wireless communication through Mesh Networks, the internal RF antenna, its capability of energy measurement in four phase quadrants, and its ability of remote energy cut and re-closing operations. For remote communication, a RF module compliant with the IEEE 802.15.4 standard 2 implementing a ZigBee stack was also included.

Communication Interface.
All smart meters manage their wireless communication with the aggregator (ZigBee stack mesh network) by acting as a router to the aggregator using a proprietary protocol [7]. Additionally they have their own digital certi cates used on the authentication process. A er the authentication process, all ABNT commands include a digital signature that has to be validated by both sides to assure a successful command execution. e digital signature con guration is hard coded in the meters and follows [17,20] A er receiving the digital certi cate, the smart meter is able to be installed in the eld and to start the communication with the MDC. Figure 3 shows the temporal structure of communication between the meter and server.

Aggregator
e aggregator is the interface between the Authentication Application, the MDC and the smart meters. It is responsible for communication protocol conversion and packet routing, acting as a gateway between the Smart Meters and the MDC system. Its main components are a ZigBee coordinator communication module and a Backhaul application. e (the Backhaul so ware) is responsible for sending routing information from the meter to the Cloud Application and vice-versa, converting protocols. It also manages the list of available meters and synchronizes the messages between the MDC application and the smart meters.

SECURE AMI STRUCTURE
e data security and authenticity is guaranteed by two layers: (1) an authentication scheme between the smart meters and the Authentication Application, which is encapsulated inside a container based on SecureCloud platform, and (2) a symmetric cryptography layer, which connects smart meters, the aggregator and the Authentication application, and is based on both Advanced Encryption Standard (AES) [19] and the authentication processes.

Authentication
e Authentication Application should assure that only authenticated meters can communicate with the MDC. Requests from non-authenticated devices should be ignored.
When a meter connects to the network, a er receiving the synchronism response, it sends an Authentication Request with its digital certi cate to the Authentication Application. e Authentication Application veri es the received certi cate with the meter's public key and sends an authentication response. A er the authentication, all data requests and responses exchanged between the meter and the application are signed.

Encryption
By default, ZigBee communication is not encrypted, so any device can connect to the network. A transport key message sent to the device has to be lled with a null key to indicate that the device should use a key saved in its memory. A er that, all the message is encrypted. e data exchanged in the network is encrypted using the symmetric cryptography AES-128 encryption algorithm (limited by Zigbee module). e meter TPM and the ZigBee module are able to process this type of cryptography. A er decrypting the Zigbee messages, the aggregator communication module forwards the received information to the Backhaul where the data is encrypted again with the AES-256 encryption algorithm.
e Authentication Application decrypts, adds a digital signature to the received data and sends it to the MDC Application. Assuming an Authentication Application located in a secure platform, the data can be sent to the MDC Application without encryption. All data originated inside the MDC Application are also encrypted by the Authentication Application towards the aggregator.

CLOUD APPLICATIONS
e Cloud Applications consist of an Authentication Application, a MDC Application and a Database. ese applications are divided into micro-services, executed on SGX containers and connected to each other via an event bus as illustrated in Figure 1.
To securely execute these applications inside SGX enclaves, three main issues must be addressed: (1) the secure data processing within container, (2) the secure data communication between containers and (3) the secure data communication to the outside [6]. Such issues are transparently addressed by SecureCloud platform since it leverages the SCONE secure container mechanism [3] to run each application inside a secure environment, use a REST communication protocol via HTTP between aggregator and the Authentication Application, and use enclave-terminated TLS connections [4].

Authentication Application
e authentication Application connects the aggregator and the MDC ensuring the authenticity of each smart meter connected to the MDC. Its rst task is to establish a secure connection between each Smart meter and the MDC, using the aggregator as a bridge.
en, it veri es the certi cate sent in the authentication request, validates it and answers with an authentication response. A er the authentication process is successfully completed, the authentication application connects by socket to the MDC application.
Once the connections are complete, the authentication application keeps both connections opened and becomes the bridge to encrypt and decrypt messages but also the veri er through the validation of the messages from its signature veri cation. It receives encrypted messages from aggregator and send them decrypted to MDC; or it receives decrypted data messages from MDC and sends encrypted data messages to the aggregator.
ere is a main process to receive the connections from the aggregator. For each connection a new thread to control the connected meter is created.

MDC Application
e MDC application consists of a TCP/IP server that opens communication channels with the authentication application to extract data from meters connected to the aggregator.
Every 15 minutes, the MDC application sends a request to the aggregator in order to collect data readings from the meters. e main function of the MDC application waits for new connections. For each new connection, a new reading thread is created and starts requesting data. A timer is used to control when the messages should be sent. When a message request is sent, the MDC waits for the answer of this command before sending the next message request. When a response is not received (timeout), the MDC should send a request retry. Upon a message reception, a er the extraction and veri cation of the necessary data, the data is stored into the Database "phasor" table. e data comprises information about voltages, currents, power and voltage angles or demands and totalizators from the meters.

DATABASE and CASCA/DB
A er the authentication process, the MDC periodically extracts the "phasor" data from the smart meter and input these data in a database. is databased is managed by our secure data management solution, CASCA/DB, described next. e use case described in this paper has strict security requirements for communication and data storage. In order to meet them while also simplifying the application development, we implemented and employed an application framework that encapsulates secure communication (CASCA micro-service) and data management functionalities (CASCA/DB micro-service). e framework named Customizable Adapter for Secure Cloud Applications (or CASCA) was conceived in order to allow quick development of secure micro-services. It provides a pool of threads working to execute con gurable tasks, a logging system, and secure sockets (SSL or TLS) for TCP/IP communications. All these functionalities are present and requested by several system servers.
e framework is also implemented over SCONE (in order to easily support code a estation) and TaLoS (in order to have all TCP/IP  communications ending inside CASCA enclaves). e data management solution (CASCA/DB) that completes the framework is based on the CASCA communication services and is itself implemented as a micro-service. CASCA/DB o ers an SQL-based querying system that stores data on a separate secure key-object distributed service. e secure key-object service is called ChocolateCloud, [21], and is also part of the SecureCloud project. CASCA/DB is responsible for translating SQL queries from the applications into key-value API calls to ChocolateCloud. e stored data is made available for use by other applications like Metering Data Management and billing systems.
For the smart meter cloud application, only CREATE TABLE, SELECT, and INSERT statements were supported by CASCA/DB. For obvious security reasons, the translating engine runs inside SGX and all statements are transmi ed over TLS connections. Also the communication between CASCA/DB and the key-object distributed server uses TLS connectors.

RESULTS AND CONCLUSIONS
e use case speci cation was a joint work (secure cloud provider, application developer's team, micro-service developer's team, energy utility managers). Despite the expertise of the team, the deployment process has been facing several challenges. For example, application developers were o en always unaware of limitations of non supported libraries. Also, performance issues related to the database population (network delays to populate a database in Europe with metering data collected in Brazil and code updates of the database storage server) had also impacted on the deployment. Such challenges are expected when integrating next-generation technologies. e initial issues are being addressed and the processes streamlined. e described applications of the use case are now fully operational and under performance tests. We are also addressing the SecureCloud platform impact on deploying and executing privacy algorithms chosen according to previously de ned role access criteria mapped to the application (for example, billing or energy demand monitoring) or user.
is document described the implementation process of the monitoring and control applications developed as use case of an AMI system for the validation of a SecureCloud platform including secure data communication between real smart meters and the MDC and metering database which are implemented as cloud applications. e smart metering application was implemented and integrated with the SecureCloud platform to evaluate its performance in real applications. e cloud platform and the application were independently developed. Its integration are currently in a validation phase. e use of the CASCA/BD framework played an important role in the development process. SGX is a new technology that introduces many hardware-level aspects that make the coding of secure application more complex. CASCA/DB abstracts most of these complexities, streamlining the development by providing e cient communication and data management services.