A Multi-stack Simulation Framework for Vehicular Applications Testing

The vast majority of vehicular applications leverage vehicle-to-everything communications (V2X) to increase road safety, optimize the available transportation resources, and improve the user experience. Because of the complexity and the high deployment costs of vehicular applications, it is usually convenient to extensively test them by simulation. We present an open-source simulation framework for the ns-3 simulator, featuring state-of-the-art vehicular communication models, in which the mobility is managed by the SUMO (Simulation of Urban MObility) simulator. Unlike other simulation frameworks, where the user is mostly limited to a single communication stack, our framework uni � es multiple stacks under a single open-source repository. The framework is designed to make it easier to con � gure the communication stacks, and to enable a fast and easy deployment of vehicular applications. It comes with the support for centralized and distributed vehicular network architectures, embedding the IEEE 802.11p, 3GPP C-V2X Mode 4 and LTE communication stacks, and with vehicular messages dissemination stacks compliant with ETSI standards. We also present two sample applications thought to show the potentiality of the framework, namely an area speed advisory and an emergency vehicle alert .


INTRODUCTION
Cooperative Intelligent Transport Systems (C-ITS) have been identied by private and public stakeholders as providing a great potential to enable the technologies required by the most ambitious vehicular applications, by allowing direct and fast Vehicle-to-Everything (V2X) communications.Two main direct communication standards have been developed to enable C-ITS services: IEEE 802.11p, the WiFi proposal for vehicular networks, and 3GPP C-V2X, coming instead from the cellular world.Both technologies can operate in the 5.9 GHz band (5850-5925 MHz), which has been reserved for vehicular communications in Europe.
The debate on which of the two protocol stacks is best suited to meet market needs is still open in the scientic community, where some argue that the solution proposed by 3GPP provides higher performance in terms of throughput and reliability.Furthermore, the presence of two interfaces, namely PC5 for Vehicle-to-Vehicle (V2V) communications, and Uu for communication with the cellular infrastructure, suggests that C-V2X is more suitable to enable reliable Vehicle-to-Network (V2N) services, also by taking advantage of the commercial spectrum of cellular network to access cloud-based applications.
Conversely, supporters of IEEE 802.11p maintain that C-V2X is still in its early development stage, with fewer oneld validation tests with respect to 802.11p.As a matter of fact, the IEEE solution was rst approved in 2009 and later amended in 2016 and, in the meantime, there have been several eld trials such as the ones in [5] and [13].Several semiconductor companies have already designed and tested 802.11p-compliant products, and several carmakers are deploying them in their vehicles [21].
In this paper, we present a simulation framework for fast and easy deployment of V2I (Vehicle-to-Infrastructure), V2N and V2V-based applications, which allows the selection of dierent communication protocols (namely IEEE 802.11p, 3GPP C-V2X, and 3GPP LTE) and provides the tools that are needed to set up basic vehicular communication scenarios using state-of-the-art simulation models.It features a standard-compliant ASN.1 (Abstract Syntax Notation 1) encoder/decoder for ETSI vehicular messages, and it is based on ns-3 (network simulator 3) and SUMO (Simulation of Urban MObility), with the former providing the model of the involved communication stacks, and the latter managing vehicle mobility.The aim of this project is to support other researchers by providing them with an open-source framework [1] that contains the communication models needed to recreate centralized V2I/V2N-based services, and purely distributed V2V-based scenarios.
The rest of the paper is structured as follows: Section 2 reviews the state of the art on literature work, comparing the available V2X technologies and already released VANET simulation frameworks.Section 3 presents our proposed ns-3-based simulation framework, and Section 4 describes two sample applications we developed on top of our framework.Then, in Section 5, the main simulation results are discussed and, nally, Section 6 provides some nal remarks.

RELATED WORK
A considerable amount of literature on the comparison between 802.11p and C-V2X, under multiple aspects, has been published in several works, such as in [2,11,14,22,24].Whilst some of the high-level analyses, such as [24], try to keep a neutral position and to show the great potential and possibilities oered by the two communication protocols, as the studies go into more technical performance analysis, there is a vast majority of them suggesting that C-V2X outperforms 802.11p.
As an example, in [22] Vukadinovic et.al. show the superior performance, in terms of reliability, of the 3GPPproposed solution.The analysis has been made taking into account truck platooning as baseline application.In [14], a simulation-based analysis points out the better performance of C-V2X both in terms of throughput and at the MAC layer.
The superior performance of C-V2X is also highlighted in [2], where the general radio link layer improvements are associated to better market conditions, stating that C-V2X may leverage the maturity of the cellular solutions standardized by 3GPP.
The advocates of 802.11p, such as [11], stress the number of tests drive already undergone by the IEEE-proposed solution, thus claiming it to be more mature and with more readiness towards its deployment.Furthermore, several works recognize that 802.11p outperforms the current implementations of C-V2X in terms of end-to-end latency [14], while others, such as [22], point out still open problems of C-V2X such as SPS persistent collisions.
The aim of this work is to provide a new tool to determine which protocol or implementation is best suited for centralized and distributed vehicular applications, and maybe clear some of the doubts still open in the scientic community.
Among the V2X-dedicated network simulators including the complete protocol stack of each communicating entity, the Veins framework [18] is surely one of the most established.It features a bidirectional coupling between the urban mobility simulator SUMO and the network simulator OMNeT++ [15], via the so-called TraCI (Trac Control Interface).Beside its native 802.11pimplementation, several other projects, such as SimuLTE [20], have been used to extend the Veins capabilities with accurate LTE models.Veins has also been extended by the Artery and Vanetza projects [17], which provide Veins with a fully functional ETSI ITS-G5 model and a exible platform for VANET application prototyping.
In this project, the TraCI interface has been used to couple the SUMO functionalities with ns-3, by using a modied version of the module available in [4].Unlike the other already available simulation frameworks, where the user is mostly   The choice of ns-3 as a basic simulation platform was also driven by several advantages it provides, including the huge community behind it and the stability of the code after many years of improvements.

SIMULATION FRAMEWORK
Our simulation framework is composed by two dierent pieces of software, that interact among each other and together provide a exible platform for application testing: • SUMO v1.6.0 [12], which allows users to model the positions of the nodes of the network (i.e., in our case, the vehicles) following realistic mobility patterns and dynamics.SUMO enables the creation of arbitrarily complex vehicular scenarios, and the interaction with the simulation elements using the well-known TraCI API.SUMO is also providing a GUI to the end-user, making it possible to visualize and interact with the simulations in a much easier way than just launching them and retrieving the nal results.• ns-3 [19] v3.29, an open-source discrete-event simulator, which allows users to model all the aspects of the communication among the various entities, including the involved network stacks.The elements of our framework are schematized in Figure 1, in which it is also possible to see the SUMO GUI during the execution of a simulation, and the dierent ns-3 modules which can be selected for the V2X application testing and evaluation.
In our framework, the message exchange is based on standardized ITS messages, dened by ETSI.Several dierent types of message, with the aim of facilitating the exchange of data between moving vehicles, are currently dened.The most commonly encountered ones, in vehicular networks, are the so-called CAM (Cooperative Awareness Message) and DENM (Decentralized Environmental Notication Message).The rst ones are messages which are periodically broadcasted by vehicles, with a frequency up to 10 Hz, containing information about the position, speed, heading and other relevant variables coming from the current status and attributes of the sender.
The second ones are instead event-based messages, which are normally sent to disseminate the information related to road hazards, possible dangerous conditions or generic events which are of potential interest for the receivers [8,10].

V2I/V2N Communication Models
In the V2I/V2N models, vehicles are congured to periodically send CAMs towards a server that collects them, runs the application logic, and sends DENMs back to vehicles, when needed.
In this case, two communication models can be used to enable the connectivity of the vehicles towards a centralized service.Depending on which one is selected, the resulting scenario can be seen as V2N or V2I.The two models are: (1) LTE, using LENA [3] as simulation framework, in which a classic LTE network is established, with the vehicles acting as UEs (User Equipment) connected to the eNB (eNodeB), that is in turn connected to the EPC (Evolved Packet Core) through the S1-U interface.The EPC implements the SGW (Serving Gateway) and PGW (Packet Data Network Gateway) blocks.The PGW is connected to a remote host that runs the application server logic.This communication can be seen as V2N, since the UE is connected to a service provider that can be ideally everywhere in the Internet.As far as the control-plane is concerned, LENA implements a model for the MME (Mobility Management Entity) node, that is connected to the eNB via S1-AP interface, and to the SGW-PGW block through the S11 interface.The main functions of the MME include the session initialization and bearer management.The resulting conguration can be seen in Figure 2.
(2) 802.11p, using the WAVE (Wireless Access in Vehicular Environment) model available in ns-3.In this case, vehicles are equipped with OBUs (On Board Units), and communicate with a remote host that is placed behind a RSU (Road Side Unit).Due to the proximity between the clients and the server, and due to the fact that the clients and server are in the same subnet, we can dene it as a V2I communication.The 802.11p WAVE framework is depicted in Figure 3.

V2V Communication Models
In the V2V models, vehicles are congured to broadcast CAMs and DENMs directly among themselves, using a direct communication.In this case, the application logic is deployed inside the vehicles, in a purely distributed fashion.
In order to enable such a conguration, two dierent communication models can be selected: (1) C-V2X, using the framework proposed by Eckerman et al., [6], in which a C-V2X Mode 4 network is modeled.In this model, the vehicles communicate through the PC5 interface and directly exchange messages with their peers through sidelink (SL), without relying on the eNB.The resulting framework models an out-ofcoverage communication scenario, and is depicted in Figure 5.

C-V2X CHANNEL
(2) 802.11p, using the WAVE stack available in ns-3.As in the V2I scenario, vehicles are equipped with OBUs, but in this case they broadcast CAMs and DENMs among themselves.The V2V 802.11pWAVE-based communication logic is depicted in Figure 6.

ETSI Facilities Layer
In its ITS architecture, ETSI has dened a Facilities layer [7] to support the distribution and processing of Application messages.The implementation of ETSI-compliant features of the Facilities layer in ns-3, adds an important aspect to the simulation framework, namely, the possibility to be used in hardware-in-the-loop scenarios.The messages generated in ns-3 can thus be received and decoded by real V2X devices, which in turn can interact with simulated entities that have the possibility to decode standard-compliant messages.These Facilities, called CA and DEN Basic Services, manage the transmission and reception of CAM and DENM messages, and are implemented in our framework following the ETSI standards on ITS messages [8][9][10].
ASN.1, which is used to encode CAM and DENM messages, allows the representation of complex data structures that can be read by any platform.This notation is completely programming language-agnostic, allowing dierent platforms, with dierent architectures, to exchange information.
Within the CA and DEN Basic Services, two modules for ASN.1 CAMs and DENMs encoding and decoding were created for our framework.These modules are in charge of receiving ASN.1-encoded messages, extracting the relevant information, and providing the ITS-S applications with the The open-source tool asn1c [23] is used to generate the encoding and decoding functions of the ASN.1 messages.
In Figure 4, we depicted the logics implemented by the CA and DEN Basic Service at the Facilities layer.In particular, the CA Basic Service allows the ITS-S applications to encode CAM messages, and, through the CAM Transmission Management, it internally handles the CAM dissemination logic.Once a CAM is ready to be sent, the CA Basic Service delivers it to the lower layers of the communication stack.At the reception side, the CA Basic Service should manage the CAM reception from the underlying layers, and decode the ASN.1-encodedCAM messages.Finally, the information included in the CAM should be delivered to the requesting ITS-S applications.The DEN Basic Service provides similar features to the ITS-S applications sending and receiving DENM messages.
Although not all the features requested by ETSI (such as security, LDM, or GeoNetworking) are present in our model so far, the core functionalities of messages encoding/decoding in the Facilities layer have been implemented.

APPLICATIONS
The simulation framework presented in this work comes with two sample applications, one for the V2I/V2N case, developed with a centralized client/server architecture, and one for the V2V case, developed in a distributed fashion.Each of them has been developed with the aim to showcase the potentiality of the framework, and feature ETSI standardcompliant message encoding and decoding.

V2I/V2N Application: area speed advisory
The application proposed for the centralized V2I/V2N communication models is based on a client/server architecture.
As far as the LTE communication is concerned, the server is placed in a remote host connected to the PGW, while it is located in a host directly connected to the RSU when IEEE WAVE is adopted.The main dierence between the two communication models, from the application point of view, resides in the higher end-to-end latency experienced when using LTE: in this case, indeed, both CAMs and DENMs need to pass through the EPC before they are delivered, while the usage of 802.11p allows the server to be directly behind the RSU, just one hop away from the clients.The map used in this context includes two road crossings connected through a central two-way street, and is depicted in Figure 7.The network access point (eNB for LTE and RSU for 802.11p) is placed at the center of the map, with sucient coverage to ensure connectivity to all vehicles traveling in the simulated area.
The core logic of the application is to divide the map in two dierent zones: the rst zone is the central one, comprising the two crossings, in which the maximum allowed speed is 25 km/h.The second zone is instead the outer one, where the vehicles are allowed to reach a speed up to 100 km/h.The server monitors the position of the vehicles, by reading the CAMs, and warns them if they enter an area with dierent speed restrictions.Vehicles are congured to periodically send CAM messages, which are received by the network access point and forwarded to the server.The server, aware of the boundaries of the two areas, analyzes the position of the vehicles.Whenever it realizes that a vehicle is moving from a zone to another, it sends a DENM message, warning the driver about the necessity of reducing his/her speed or the about possibility to increase it.
Figure 7 shows the V2I/V2N sample application running in SUMO, with ns-3 managing the communication and application logic.In particular, it is possible to see 4 vehicles, two in the high-speed area (veh1, veh2) and other two in the low-speed area (veh3 and veh4).In this case, once a vehicle receives a DENM informing the driver to increase the speed, ns-3 tells SUMO (through TraCI) to change the vehicle color to green and increase its speed.Conversely, whenever a vehicle moves from the high-speed to the low-speed area, its color is changed to red and its speed is reduced.
In this scenario, CAMs are sent as unicast from the vehicles to the server running the application logic.At the same time, the server generates and sends unicast DENMs to notify the vehicles changing speed area.

V2V Application: emergency vehicle alert
Other than the aforementioned V2I/V2N application for the related models, the V2V models are showcased through an Emergency Vehicle (EV) use case, supporting both IEEE 802.11p and 3GPP C-V2X Mode 4 for vehicle connectivity and relying on ETSI CAM and DENM messages.In particular, in either case, ETSI messages are directly exchanged between vehicles.As in the previous case and in accordance to the standard, all vehicles exchange CAM messages to inform nearby nodes about their current status.The scenario also includes, however, the presence of EVs, which should be able to travel without being slowed down by the other vehicles.An EV could be an ambulance, a police motorcycle, or a reghter vehicle.In particular, we modeled a scenario in which two EVs are periodically sending DENM messages to all nearby vehicles.Upon reception of a relevant message (i.e., a DENM message from an approaching EV), a normal vehicle will try to limit its hindrance to the EV.If it is travelling on the same lane as the EV, it will speed up, and try to change lane as soon as possible.If on a dierent lane, it will slow down.In this way, the EV will be able to pass a normal vehicle without being forced to reduce its speed.
The map used in this case represents a urban scenario, where normal vehicles have a maximum speed varying between 30 km/h and 60 km/h and EVs can, instead, travel up to 75 km/h; a circular road segment is involved, with two lanes for each direction of travel.A screenshot from SUMO while running the application is presented in Figure 8.In particular, the red vehicle is an EV broadcasting DENMs, while the orange vehicles (like veh1) are nodes that have correctly received and parsed a DENM from the EV and are slowing down to be safely overtaken.The green vehicle (veh2) is instead a node traveling on the same lane as the EV.It is thus trying to accelerate to change lane as soon as possible and let the EV go by.Finally, the yellow vehicles (such as veh3) are cars which do not need to react to the approaching emergency vehicle, as they may be traveling in the opposite direction, they might not be directly interested by the EV trajectory or they may be still too far away.

SIMULATION RESULTS
Both applications are evaluated in this section.As the main goal of this paper in not to compare in details the main communication technologies (namely, LTE and 802.11p for the V2I/V2N case and C-V2X and 802.11p for the V2V case), but to show the potentialities of the complete framework, we focus on algorithm-based metrics, leaving a more detailed evaluation of the technology-related KPIs, such as latency and Packet Delivery Ratio, for a future work.Such a detailed evaluation would require, in fact, extensive simulations over selected use cases, varying the several parameters of the involved communication stacks.

V2I/V2N Application
We evaluated the performance of the V2I/V2N scenario focusing on a safety situation.We introduced two modied trac lights always showing a green light to incoming vehicles.In this way, the vehicles will reach the intersections without slowing down due to the trac lights, and, in some cases, they will be unable to stop and avoid a collision.We then observed the average number of collisions with and without our area speed advisory system.
Since our goal is to provide a sample application to show the potentialities of the framework, a collision avoidance scheme, in this context, has not been implemented; this solution also allows us to dene a metric (i.e., the number of collisions when no trac lights are regulating the trac) to evaluate our sample application, both when a speed reduction is mandated through DENMs and when, instead, the algorithm is disabled and all the nodes can keep their maximum speed.
In order to evaluate the application, we tested it with several scenarios, involving dierent numbers of vehicles.In particular, we varied them between 5 and 40 vehicles, and we plotted the results as a function of the number of vehicles per kilometer.The vehicles could reach a maximum speed of 100 km/h, and for each density we ran 100 simulations, each lasting 200 s.
The results are plotted in Figure 9, in which the dierent vehicle densities are reported on the x axis and the average number of collisions at road crossings on the y axis.For each value, the 95% condence interval on all the simulations is plotted.The results show the benets of the speed control application on the total number of road crossing collisions.In particular, the simple application developed on top of our framework, which basically slows down the vehicles approaching the intersections, was able to avoid a large number of collisions.The average number of reported collisions, when the algorithm is disabled, increases up to 8 collisions per simulation, when the density is between 11 and 12 veh/km.Then, due to the increased trac, and to the consequent drop in the average vehicles speed, the total number of collisions starts to decrease.Instead, when the algorithm is enabled, the number of collisions is always low, with an average of less than 1 per simulation, even when the density is high.

V2V Application
As the main metric for the assessment of the V2V-based application, we focused on the average speed of the EV, under dierent trac conditions, comparing the case in which the alerting is enabled (i.e., DENMs are sent by the EVs), either through 802.11p or C-V2X, and the one in which it is disabled.
As before, in order to properly evaluate the proposed application, we considered dierent vehicle densities, ranging from a total of 3 veh/km up to more than 13 veh/km.For each density we ran 100 simulations, each lasting 200 seconds, always using dierent mobility traces, each one including two EVs, one per travel direction.The speed of the two EVs has been averaged over all the simulations.The results are depicted in gure 10, in which the dierent vehicle densities are reported on the x axis and the measured average speed on the y axis.For each value, 95% condence intervals over the 100 simulations are reported.
As is possible to infer, the results show how the proposed solution can be an eective, yet simple, algorithm for an EV use case, and how our framework can be eectively used to assess standard compliant applications for V2V.Indeed, the EVs are almost always able to keep their maximum speed when they are sending DENMs to nearby vehicles, no matter the trac density or pattern.Instead, when no DENMs are sent and the vehicle density starts increasing, a quite consistent speed drop (sometimes more than 10 km/h) can be observed.
It is also possible to notice how the selected V2V technology does not make a signicant dierence when it comes to the algorithm eectiveness, at least as long as all the needed DENMs are correctly received by the involved vehicles.

CONCLUSIONS
In this work, we presented an open-source framework for fast and easy deployment of V2X applications.The framework, which is based on the ns-3 and SUMO simulators, features state-of-the-art models for V2X communications, and standard-compliant vehicular message dissemination stacks, including the ETSI CA and DEN Basic Services.The framework provides exibility in the application conguration, and can be used to easily switch the communication technology and the network architecture.
Thanks to the interaction with an up-to-date version of sumo-gui (i.e.SUMO's graphical user interface), it provides an easy way to visualize the simulations and, possibly, also debug them.ns-3 is also representing a stable and wellmaintained network simulator, constituting a solid base for our V2X simulation framework, thanks to its community and to the several state-of-the-art models which has been and are currently being developed, such as the 5G version of LENA [16].Two sample applications are presented: area speed advisory as a centralized service, and emergency vehicle alert as a distributed service.They represent a baseline for the user, which can use them as examples to build his or her own scenarios, leveraging on functions which simplify the management and usage of the ETSI CA and DEN Basic Services.
The simulation results show, for both applications, a signicant improvement in the measured KPIs.In particular, the area speed advisory application increases the safety at road intersections, by forcing vehicles to slow down when approaching more critical road segments.The application has been proven to avert a high percentage of collisions, even in a scenario in which the vehicles approach the crossings without slowing down.The emergency vehicle alert application, instead, is eective in giving priority to particular classes of vehicles.Namely, it allows them to keep a higher average speed, with respect to the case in which the service is not active.In our toy scenario, two emergency vehicles were able to keep almost their maximum speed even when the density of the other vehicles was relatively high.
Those applications, although very simple in their core logic, demonstrate the potentiality of our open-source simulation framework, and the importance that vehicular applications will have in the future of ITS.

Figure 1 :
Figure 1: Main components of our simulation framework.

Figure 7 :
Figure 7: Two screenshots from SUMO-GUI, showing the map and the operation of the sample application for the V2I/V2N framework.

Figure 8 :
Figure 8: Two screenshots from SUMO-GUI, showing the map and the operation of the sample application for the V2V framework.