Networks of Trusted Execution Environments for Data Protection in Cooperative Vehicular Systems

. Networks of autonomous vehicles roaming in smart cities raise new challenges for end-to-end protection of data in terms of integrity, privacy, e(cid:30)ciency, and scalability. This paper provides a survey of Networks of Trusted Execution Environments (NTEE) architectures. NTEE combine the strong, hardware-rooted security guarantees of the TEE deployed locally in the vehicle, with the distributed protection of a decentralized consensus protocol. We identify three main families of consensus protocols and analyze their architectures, performance, and security, including improvements brought by the TEE. Overall, voting protocols tend to be more e(cid:30)cient for smaller networks, while lottery-based schemes are not easy to apply in a vehicular context due to higher overheads. Both types of protocols reach an intermediate level of security, with variations in byzantine tolerance and types of threats. Graph-based protocols tend to achieve both e(cid:30)ciency and (cid:29)exibility in terms of network topology support, but their security still remains to be explored.


Introduction
Networks of autonomous vehicles roaming in smart cities raise new challenges for end-to-end protection of data. Such hyper-connected, decentralized systems produce and collect from the environment an increasing amount of data, analyzed, stored, and shared with the various stakeholders of the vehicular ecosystem. Examples include itinerary, speed, passenger personal data, network state, or road trac conditions. Such data is unfortunately highly vulnerable. Protection should strike the right balance between data integrity, intimately related to safety to avoid hazards in autonomous decision-making, and privacy to minimize personal data collection [21]. Protection should also be practical and scalable, supporting dierent network topologies, both centralized and decentralized.
NTEE provide both a local secure environment with strong security rooted in hardware (e.g., trusted execution, secure storage, attestation) and decentralized protection such as distributed data integrity. However, it remains unclear which class of protocol matches the previous goals and related trade-os.
A number of comprehensive surveys already addressed some elements of the problem for instance, on attacks and counter-measures for vehicular networks (e.g., [22,23]), on consensus protocols, either from a broad perspective [36], including TEEs, or from an IoT standpoint [20], but not specically for autonomous vehicles, or through more specic studies on blockchain applications for automotive [29]. However, we felt the need for a shorter, more focused, overall view giving a preliminary assessment of NTEE architectures for protecting data in networks of autonomous vehicles, highlighting research trends and challenges from a literature review, to be conrmed by simulation results in a second step.
Several families of well-known approaches also provide elements of solutions, such as for V2X isolation and trust management (e.g., in-vehicle ECUs isolation, trusted computing [15], often TPM-based, remote attestation, HSMs [35] to guarantee strong, certied security proles for vehicle or roadside units [10,11]), privacy (e.g., PKIs [16], pseudonym systems [30]), and resilience [2,5]. Those solutions are already well investigated, but beyond the scope of this paper. This paper surveys existing NTEE architectures for cooperative autonomous vehicles, combining the elds of TEEs and consensus protocols. We distinguish three main families of protocols, voting-, lottery-, and graph-based. We discuss their architectural, performance, and security properties. We also show how such properties may be improved using the TEE to meet vehicular constraints, e.g., enhancing overall eciency for distributed computation of trust [8]. 2 System Overview and Approach NTEE are distributed systems, composed of nodes, each hosting a TEE, and coordinated by a distributed consensus protocol [7]. The aim is to reach agreement between all participants in the vehicular network. Dierent network topologies are possible (see Fig. 1). We assume the network provides notably 5G connectivity, including to the infrastructure, e.g., trac lights, smart city sensors wireless connectivity is also possible, e.g., for vehicle-to-vehicle interactions.
The NTEE landscape may be captured using the following taxonomy (see Fig. 2). For each class of consensus protocol, we analyze its inherent architecture, performance, and security properties. Architecture We identify three families of protocols sharing similar architectural features. For each family, we sketch the protocol principle, and discuss its scalability, dynamicity, and exibility to support several network topologies.
Traditionally, the weak synchrony assumption is made: messages may be delayed, duplicated, delivered out-of-order, or lost. They could also be forged by byzantine nodes. In Vehicle-to-Everything (V2X) networks, the intermittent synchrony seems more relevant: messages are sent on average within bounded time, while allowing this constraint to be relaxed during some short periods [26]. Several network topologies should be supported by the NTEE, ranging from centralized (e.g., star networks), decentralized (e.g., mesh networks), to hybrid. Switching topologies may be possible depending on the vehicular environment or trac: In smart cities, vehicles may easily connect to the infrastructure and/or network, as coverage is dense in urban areas. Trac may be managed smoothly mostly in a centralized manner, with also strong locality (e.g., 4G/5G femtocells, edge technologies).
In city borders, vehicles may reach the infrastructure through another vehicle.
Both centralized and decentralized topologies are possible.
In scarce connectivity environments, such as rural areas or zones where 5G infrastructure has not yet been deployed, topologies tend to be fully decentralized, as autonomous vehicles must coordinate peer-to-peer. Security and privacy Condentiality is ensured by how nodes access the distributed ledger, either permissioned or permissionless. This property is related to the network architecture, and to how easily new participants may be added in the consensus protocol.
Condentiality may be guaranteed through encryption, before uploading data to the distributed ledger; or through isolation by sharing data on a private ledger only between a selected subset of participants. Encryption also allows sharing private data using public key cryptography, but may not be applicable to some IoT devices due to the induced overhead. Current ITS PKI systems also issue multiple public keys for the same ITS station, which could as well severely limit performance. For isolation, in some cases, nodes not part of a private group may be allowed to take part in the computation to reach agreement.
Distributed integrity of data manipulated by NTEE nodes is also needed.
Blockchain protocols have notably been used for providing vehicular data immutability [20]. Reputation mechanisms within vehicles are also useful for event linkability and anonymity [34].
Availability is essential for reliable cooperation between vehicles, e.g., for exchange of information and decision-making. Fallback mechanisms are needed when the vehicle cannot reach the network nor the infrastructure [2], due for instance to communication or back-end systems failure, in local MEC or cloud servers. Solutions include bounded-time recovery [13] or self-stabilization [14] to guarantee the network of vehicles stays in a safe state. Failures due to interpretation by the vehicle of its rapidly changing environment, added to internal software and hardware aws, frequent real-time updates, and complexity of multiple autonomic loop orchestration are still major challenges ahead. The platooning case has been particularly investigated in terms of solutions [5].
Privacy is a key challenge in vehicular networks, guided by a principle of data minimization. The aim is to regain control over data collected, shared, and used in the large, multi-stakeholder V2X ecosystem, to protect vehicle identities, avoid vehicle tracking, and preserve driver and passenger attributes against unauthorized nodes [21]. The full spectrum of Privacy-Enhancing Technologies (PETs) is applicable as counter-measures. Homomorphic encryption, secure, veriable, privacy-preserving multi-party computation (e.g., for committee-based proofof-stake [36]), dierential privacy, hardware protection mechanisms, partial observability, pseudonymity and anonymity techniques are just but a few. While currently deployed V2X solutions mostly rely on the use of pseudonyms [3,30], a number of consensus protocols are essentially based on some form of leader election mechanism. This might be a challenge for vehicular networks, where nodes constantly change identities. A secure multi-party computation scheme could help to implement the election algorithm in a privacy-preserving manner. Promising solutions for the platooning case have notably been explored [31].
Sybil attacks could be used to hijack such protocols based on cooperation: an attacker could generate any number of ghost vehicles in its neighborhood to vote for him. Available counter-measures are either based on resource testing, location or position verication, or encryption and authentication [18]. 3 Voting-Based Consensus Architecture Participants vote to elect a leader in charge of executing a command on the distributed ledger. Practical Byzantine Fault Tolerance (PBFT) is the rst voting consensus working with weak synchrony assumptions [12]. Since then, many variants have been proposed [24,26,36,37]. To reach agreement, PBFT is based on three rounds of message exchanges (pre-prepare, prepare, and commit). This guarantees that commands are atomically executed and strictly ordered, resulting in a nal-state consensus.
Voting-based consensus protocols are generally considered to have limited scalability in terms of number of nodes [17]. Scalability was also not much explored beyond n = 10 to n = 20 nodes [33]. Informally, every participant joining the network has to be acknowledged by all others. Adding (or removing) nodes scales as O(n 3 ), due to such intensive all-to-all network communications. Such protocols could therefore be very expensive for V2X where nodes are highly mobile, continuously switching from one area to another. Scalability may be greatly improved with optimizations such as canonization of phases enabling pipelining, communication complexity in each phase being reduced from O(n 2 ) to O(n) [37].
In terms of topology exibility, PBFT schemes are expected to be ecient in environments with scarce network coverage: agreement is reached rapidly when the number of participants is kept small. In an urban environment, they could be applicable to manage trac at intersections (e.g., smart trac lights) to take fast decisions. They seem less relevant in smart cities when there are many vehicles to coordinate. Performance Latency depends on the time to perform a commit on the ledger. Thus, as for bandwidth, it is ecient while the network size is kept small. Due to the O(n 2 ) message complexity, such protocols are not applicable for much larger networks. Voting-based schemes achieve medium energy eciency: while most of the energy costs are spent in communication, many messages still have to be exchanged to reach agreement. Security Such protocols are immune against downgrade or rollback attacks [9]: voting consensus are nal-state. Thus, any change committed on the distributed ledger is denitive.
Regarding byzantine tolerance, PBFT protocols are proven to tolerate up to 1/3 of byzantine nodes [33]. The main security threat is DDoS against the leader. The root cause is the protocol design itself, which is at some points centralized around the leader. The approach of choosing a new leader after a timeout expiration is only crash-tolerant, and not byzantine-tolerant, as the DDoS attack can follow the new leader. Apart from a rapid leader change, there seems to be no evident counter-measure [37]. TEE impact Security and byzantine tolerance are improved. The TEE shifts trust to the hardware, instead of the decentralized protocol. Enclaves guarantee integrity of the executed code using hardware protection. Mechanisms are also available to attest run-time integrity [1]. Moreover, by running a simple crashtolerant algorithm (e.g., Paxos, Raft) inside an enclave for every participant, a behavior similar to a byzantine-tolerant algorithm may be achieved [24].
The TEE also improves performance by reducing the number of messages needed for agreement. Most TEEs support monotonic counters, theoretically impossible to reset, which guarantee a trusted order for messages: the message number is signed together with the message, allowing honest nodes to sort them, even over unreliable networks. This approach has been explored in systems like FastBFT, where message complexity is reduced to O(n) [24]. It can also be used to improve byzantine tolerance [32]. 4 Lottery-based Consensus Architecture Instead of a vote, the leader is selected by a shared lottery algorithm in which all nodes participate. When a solution to the lottery is found by a node, it is sent to all for validity checking. Assuming a majority of honest nodes, an agreement is reached on the data sent by the winning node. The data must be consistent with the ledger current state, or it will be rejected. Each node then adds this data to its local copy of the ledger. Many algorithms are available, known as proof-of-X, e.g., proof-of-work (PoW), proof-of-luck (PoL) [27], proofof-elapsed-time (PoET) [19]. Hybrid lottery-based solutions may also choose a group of nodes that then elect the leader using a PBFT-like scheme.
Lottery-based protocols are generally more scalable than their voting counterparts despite many variations among the wide range of proof-of-X schemes. For PoW, a new node does not need to register anywhere, and can join the network just by working on the proof. In other schemes (e.g., based on cryptographic sortition [25]), a new node has to broadcast its public key to all others to be part of the selection process, which makes them slightly less scalable.
Lottery-based consensus protocols are also applicable to many topologies.
They are thus already used for distributed management of vehicular data [25].
Performance Due to latency, the applicability of such protocols may be limited for cooperative vehicles: to reach agreement and execute a command on the ledger, the lottery has to be won by a participant. The frequency of winning results should be very high if frequent updates are needed. Nevertheless, PoL and PoET might provide faster agreement between vehicles than PoW. Bandwidth eciency is highly variable, depending on the block generation rate, ranging from weak (PoW mining) to moderate (PoL).
Energy eciency can be very poor depending on the lottery mechanism. For intensive hashing algorithms (e.g., PoW), there will be a huge waste of energy to reach agreement. With other mechanisms, when the lottery process is based on the participants public keys instead of a looping computation, the energy issue is less signicant.
Security A rst threat is the occurrence of forks: two participants may win the lottery in a short time frame, which may split the ledger into two valid parts when the two results are broadcast simultaneously. Forks may be resolved probabilistically over time, by nodes choosing always the longest chain when the next result is found.
An attacker may also attempt to downgrade the distributed ledger state to a previous one to take control over the latest transactions (rollback attack). Mitigation includes using a nal state consensus, avoiding transient unsafe states, so that previous states may not be restored. A trade-o must be reached between consistency and partition tolerance, and must be considered in the NTEE deployment policy [17]: sealing blocks will prevent rollbacks, but will also prevent the distributed ledger from merging after a fork. Conversely, to avoid forks, rollbacks must be allowed.
Another threat are majority attacks: integrity holds if less than 1/2 of nodes are byzantine. If an adversary takes control of more than 50% of hashing power, consensus may be corrupted by forging blocks with double-spending transactions, or putting transactions in arbitrary orders. There is no real applicable countermeasure over public networks.
TEE impact Energy eciency may be improved by delegating trust to a secure enclave rather than to a hash function. Compared to PoW, Intel PoET randomly backs o for an exponentially distributed period of time. The TEE provides hardware attestation that the node has really awaited this time. The PoL approach is similar: all nodes draw a random number in a periodic time slot. Eective randomness is guaranteed by the enclave.
Delegating trust to hardware is also benecial for performance: time-and energy-consuming computing tasks related to proofs-of-X could be highly improved in an enclave, e.g., that may eciently attest for an amount of work [19]. 5 Graph-based Consensus Architecture New consensus protocols based on directed acyclic graphs (DAGs) are emerging. For instance, the hashgraph proposes a virtual voting consensus [6].
Nodes vote "virtually" for one another, without exchanging specic agreement messages. The DAG structure represents the inner network in memory vertices being the network nodes, edges the network links, and ows through the graph the unidirectional message communications. The hashgraph advertizes all events in the network. Nodes become interchangeable in terms of knowledge about what happens in the network, hence the virtual voting consensus. Graph-based consensus protocols appear to achieve the greatest exibility in terms of support of network topology, from centralized to decentralized. Such protocols could therefore be applicable to all network topologies found in vehicular environments.
Performance Virtual voting does not require to exchange messages to reach agreement, which should result in low latency. Embedding the consensus metadata in the gossip protocol, responsible for data exchange, is foreseen to achieve very ecient bandwidth usage, close to the theoretical limit, i.e., sending only the transaction data [6]. Such protocols should also be very energy-ecient, reaping benets from both voting-and lottery-based approaches: energy is only spent on communication, and network usage is limited thanks to virtual voting. Such a property is particularly interesting for cooperative autonomous systems.
Security Graph-based protocols achieve fair byzantine tolerance, being resilient to 1/3 of byzantine nodes and to forks. Another promising property is resistance to DDoS due to the fully decentralized architecture. How the TEE could improve performance and security of a DAG-based consensus protocol remains an open area for further research.  Figure 3 shows some broad trends for the consensus families, focusing on the performance vs. security trade-o. Security is captured through byzantine tolerance, and resistance to other attacks. In a rst step, performance is assessed through latency, critical KPI for V2X safety a ner evaluation, e.g., taking into account bandwidth and energy eciency is left for future work.
Overall, voting protocols tend to be more ecient for smaller networks, while lottery-based schemes are not easy to apply in a vehicular context due to higher overheads. Both types of protocols reach an intermediate level of security, with variations in byzantine tolerance and types of threats. TEE usage also improves security and performance. Graph-based consensus protocols are promising, notably the virtual voting of hashgraph, as they seem to achieve both eciency and exibility, but further investigation is needed on those systems.
For V2X, the three NTEE families are applicable to protect data, depending on network topology, infrastructure and use case requirements. Previous ndings will need to be conrmed by simulation results, realistic use cases, or practical deployments for coordination of vehicles. Dierent classes simulators could be explored, e.g., for vehicular trac [28], or V2X ETSI ITS-G5 [4]. NS3 is also promising as it allows to clearly separate the consensus layer from other protocol layers thanks to Direct Code Execution.
We expect exibility in the data protection architecture to be needed to support multiple consensus protocols including transparent protocol switching at run-time. Typical examples include: (1) vehicles moving between environments (e.g., smart cities, city border, rural area); (2) supporting other verticals (e.g., drones, robots, etc.); (3) aiming for dierent real-world use cases. We are thus currently implementing a modular simulation framework supporting multiple protocol families (e.g., PBFT-like, PoL, PoET) and run-time features.