A Monitoring System to Measure the Impact of a Network Application in a 5G Network

Network monitoring is an essential feature of management for most digital service providers, since it allows for evaluating the network status and its integration with a service. With the introduction of 5G Networks and implementation of solutions using proprietary 5G Cores, obtaining monitoring data can be challenging. Not only on account of legal aspects, but also due to limitations regarding support or the means by which network monitoring data is provided. Thus, a need for agnostic monitoring solution arises, being the main focus of this work. As such, this paper describes the design and implementation of an agnostic monitoring solution that relies on Software Defining Networking mechanisms. Our monitoring system was developed through open-source technologies and allows for gathering network metrics and expose them via a REST API and a Web UI.


I. INTRODUCTION
A reliable network should be able to guarantee availability and quality of service.To ensure these characteristics, it is important to have data regarding the current state of the network, allowing to conclude on connectivity issues or even foresee them.This is possible when a network monitoring solution is implemented.Not only that, traffic data analysis can prove to be a basis for implementing security features, such as anomaly detection, or traffic engineering.
As the concept of 5G Networks is increasingly popular and its promised features hold a lot of potential, many carriers have made significant advances developing and commercializing 5G Cores.This way, many services are evolving towards being 5G ready.However, implementing monitoring features in this scenario can be challenging.Since most of the available 5G solutions are proprietary, their inner workings are not always disclosed to the client, and the available information is often hard to take advantage of.The lack of documentation and easy integration are big set-backs for integrating the network Core with monitoring software.Moreover, licenses are needed to better understand said solutions, and it is common for these solutions not to have developed all of planned functionalities, given their relatively short lifespan.
This way, 5G Cores often act as a "black box" to the entities who implement a third-party 5G Network, posing as an adversity for their integration with monitoring solutions.
Furthermore, one must also consider that the 5G Network Infrastructure may be extremely heterogenous and depends Fig. 1.Monitoring and optimization workflow for the O-RAN architecture, suggested in [1] on hardware-equipment from different vendors.Open Radio Access Network (RAN) (O-RAN), for instance, is an industry-wide standard for the RAN interfaces that supports the interoperation between hardware equipment and software from different vendors.This way, O-RAN presents itself as being a disaggregated approach to deploy midhaul and mobile fronthaul networks built on cloud native principles [1].
Besides O-RAN, Open Air Interface (OAI) is another initiative to reduce the need for specific hardware equipment in the 5G Networks landscape.As such, this initiative aims to develop a full-blown 5G cellular stack on commercial off-theshelf (COTS) hardware [2].Moreover, since OAI follows the O-RAN architecture, these two technologies can be employed together, for instance, by having OAI gNodeBs (gNBs).
The existence of such approaches contributes to the heterogeneity observed in most 5G Networks, and leads to a more troublesome monitoring process, as it is needed to cope with all this heterogeneity.Furthermore, O-RAN and OAI still do not provide sufficient monitoring mechanisms, which led to Polese et al. suggesting the existence of an independent Monitoring Stack to collect metrics on O-RAN nodes [1], as seen in Figure 1.
One possible approach of retrieving network data and handling traffic is utilizing Software Defined Networks (SDN).SDN allows for more customized network experience and increased programmability, while separating the data and control plane.As a result, traffic originating or destined to the 5G Core can be handled by employing SDN.This allows for an easy manipulation of flows and efficient measurements of data, which ultimately is the motivation behind our monitoring solution.
Regarding the paper's structure, we start by addressing the Fig. 2. Positioning of the monitoring solution applicability of our system, as to provide a better context on our work -Section II.Then we move on to the state-of-the-art in regard to network monitoring solutions and methodologies, in Section III.After these sections, we start describing our solution's architecture -Section IV -and the way we implemented it, in Section V.After describing the architecture and the implementation of our monitoring system, we follow up with its evaluation, presented in Section VI.Finally, Sections VII and VIII, showcase the conclusions of our work and the future work that must be addresses to improve our solution, respectively.

II. APPLICABILITY
The work we now present was developed under the 5G Application & Services experimentation and certification Platform (5GASP) Project.5GASP is an European Project that aims to push the development of services relying on Network Function Virtualization (NFV).It assembles a consortium of diverse European companies and research institutes with experience in the technological areas addressed by the project and aims to expedite the implementation and validation of Virtual Network Functions (VNFs) in an automated way.In this context, several validation mechanisms have been developed [3] [4] [5].However, it still lacked a way of monitoring and measuring the network traffic associated with a Network Application.Thus, the developed monitoring solution aims to provide the means to cope with this restriction.Our solution may be deployed in any location given that is placed between the end User Equipments (UEs) and the Network Application and must be part of the Transport Network of a given Network Slice. Figure 2 showcases how our solution can be deployed in order to achieve the described end-architecture.
In Figure 2, our monitoring solution is deployed between the Virtual Infrastructure, where a Network Application is deployed, and the 5G Core, being part of one or several Endto-End (E2E) Network Slices, and gathering metrics from the 5G Transport Network.Furthermore, our monitoring solution is capable of monitoring both the traffic between the Network Application and the UEs and the traffic that flows between the Network Application and the Internet.This way, our solution can provide the means of accurately collecting network metrics related with a Network Application, which will then contribute to the development of improved Network Application validation methodologies and systems.
Furthermore, our solution can also be seen as an alternative approach for Telecommunication Providers to monitor 5G network traffic.This is due to the fact that many Telecommunication Providers are bound to vendor-specific 5G hardware.As such, many times, this hardware does not provide all E2E measurements that the network operators wish for and thus does not suffice the needs of Telecommunication Providers.

III. RELATED WORK
Since most Telecommunication Providers' 5G Cores are proprietary, most commonly the operators suffer from several restrictions when trying to automate different aspects of their infrastructure.This was the case in the 5GASP Project.Most consortium 5G Cores lacked the ability of providing network monitoring data through a programmatic way.While it is possible to observe the network's health and monitoring data, the information is only presented via a Web Application, which toughens the possibility of monitoring and automating network management on the infrastructure.As such, the impact of the VNFs deployed in an infrastructure is hard to quantify via software and export to the developers for validation, since the network status at the time of validating is not monitored.
As the relevance of 5G Networks rises, the number of 5G Network providers as well as 5G ready solutions accompanies this growth [6].However, these concepts are not yet fully implemented, which limits the functionalities they offer, despite the standardization of their components.As an example, the Network Data Analystics Function (NWDAF) is one of the components defined in mobile communication system standards that would be very beneficial for monitoring purposes.It derives analytics from the network and provides that data for other network functions to use [7].Another useful component is the Network Exposure Function (NEF), which is also not yet implemented in many 5G Core solutions.Since it offers the ability to track User Equipment related events and expose that information along with analytic reports to external consumers, network monitoring can be difficult to achieve in some scenarios [8].
One of the features made possible with 5G Networks is the concept of Network Slicing.This network architecture allows the creation of multiple virtualized networks over the same infrastructure, supporting different applications with distinct Quality of Service (QoS) parameters [9].Moreover, despite originating from the same 5G Network, slices provide endto-end isolation.This is a key feature to guarantee QoS requirements, since it exclusively allocates network resources to each user [10], as seen in Figure 3.
Together with the development of 5G technology, Software Defined Networks have gotten increasingly popular as well [11] [12].SDN allows for control and data plane separation, reduce the complexity and cost of network equipment, since they mostly rely on virtualization, and, thus, increase the programmability when handling data.This way, it is important Fig. 3. Example of network slices implementation [18] to note that SDN, Network Function Virtualization and 5G are complementary technologies [13].
Most of the functionalities provided by SDN are possible due to the usage of the Controllers: units responsible for managing SDN equipment in a network.Not only they configure the rest of the devices, but they are also an essential piece for the SDN equipment monitoring, since they are capable of retrieving data from every SDN device via control plane.Many SDN Controllers were already developed and are still being improved, like ONOS, Ryu1 , OpenDaylight2 and others, although the first two can be considered the best-performing ones [14].
With the rising need for reliability and high availability of network systems, many monitoring platforms and tools were already developed.Some of them consist of light frameworks that are capable of retrieving network data of many devices while producing low traffic load and maintaining a simple interface such as SNMP and Netflow.The last one, integrated with OpenFlow, also provides a reduced overload in the network, but it only retrieves flow statistics [15], while the first is able to obtain performance metrics.Other monitoring tools are more complex and offer a bigger variety of data, measuring network traffic via the network interfaces of the host in which they were deployed, aggregating those measurements as well as performance metrics from said host or separate services deployed in the same server.This is the case of Zabbix3 , a monitoring server capable of delivering performance and network metrics, integrating alarms and automatically discovering new devices [16].Also open-source and performing a similar set of features is Prometheus4 , currently one of the most promising monitoring solutions due to its powerful queries, easy visualization via Grafana5 integration and efficient storage, which make it the preferred monitoring solution for cloud computing scenarios [17].

IV. ARCHITECTURE OVERVIEW
The proposed network monitoring solution relies on the incorporation of an SDN equipment running OpenvSwitch (OVS) 6 acting as a man-in-the-middle.It was designed to be effective when integrated in both traditional or 5G Networks.To be compatible with a variety of network providers, namely private 5G Core implementations, it must not rely on information provided by the Core in order to provide accurate metrics, as this information can be limited or undocumented.Implementing a OVS-running device for monitoring purposes allows for maintaining the connectivity from the network core with other services, via OpenFlow, while analyzing packets and retrieving network traffic statistics.
Concerning the data retrieval capabilities, our solution operates with certain restrictions.Due to limitations in our 5G infrastructure, the solution does not discriminate between network domains.More precisely, the UEs' traffic that targets the Virtual Infrastructure is forwarded from the RAN to 5G Core.Given that the SDN Switch offers connectivity to the 5G Core, it consolidates the traffic originating from both the RAN and the Transport Network (TN).Consequently, our solution cannot discern the specific network domain to which the collected metrics pertain, treating all network domains uniformly.
The statistics retrieved by the controller are later processed by a custom service, the Metrics Exporter.This component is responsible for generating metrics from the data received and export them to a Metrics Collector or other third-party solutions.Both the SDN Controller and Metrics Exporter must be deployed in devices that have connectivity to each other and, individually, to the SDN equipment and the Metrics Collectors, respectively.
Another advantage of using a Traffic Analyzer detached from the network core or other services, is that the monitoring solution can be implemented in cloud computing scenarios, without requiring any further configuration on the instances of said cloud.In this scenario, the Network Applications can be deployed on a Private or Hybrid Cloud and accessed via a 5G Network.Thus, our solution works as a system that provides network performance metrics without intruding any of the parties generating or receiving the traffic to be analyzed.An illustrative architecture can be found in Figure 4.
In terms of infrastructure, the sole requirement that impacts other parties is the need to configure the egress interfaces of both the network core and cloud service provider such that they can establish connectivity with the traffic analyzer, ideally through Layer 2, in order to minimize the impact on network performance.
Besides monitoring access to the 5G Network, the solution can also retrieve internet access metrics from both parties.Given that traffic to the Internet involves accessing external networks, it is the only scenario where the 5G Core and Cloud Provider establish Layer 3 connections.The required configuration is also implemented at the SDN Switch, and Fig. 4. Generic architecture overview the data retrieval mechanism is identical to the 5G Network monitoring process, described above.

V. IMPLEMENTATION
The device implemented to enable network monitoring was an SDN Switch running OpenvSwitch and PICOS 7 .As for the controller, a Ryu instance was deployed in the switch's management network, as it provides a lightweight method of interaction.Furthermore, the fact that the SDN equipment is not meant to be reconfigured regularly, the data retrieval process is the key goal with the controller, and its swiftness is crucial.Since the solution was implemented to test network traffic involving a cloud platform or virtual infrastructure, the implemented tool for aggregating metrics was Prometheus.This decision relied on the fact that the majority of the NFV community is employing Prometheus to offer Time Series Databases.For instance, in OSM, the Monitoring (MON) component relies on Prometheus to store all the collected VNF metrics.Thus, we chose to be aligned with the NFV community and decided to use Prometheus to aggregate and store the collected network metrics.Moreover, Prometheus also comprise a vast number of plugins and is easily integrated with other meaningful tools, such as Grafana.
To scrape and process the network metrics from the SDN switch, a dockerized custom service was developed.The processing method periodically consumes statistics from the Ryu controller and converts them into usable metrics.For monitoring purposes, the service also opens an HTTP server exposing said metrics, which is configured as an endpoint to scrape in Prometheus.Regarding exportation for other users, two endpoints were created with FastAPI8 which provide data in two different formats.The first consists of a JSON file containing metric names, labels, timestamps and the respective values, while the second returns a PDF presenting graphs with the metrics in a timeseries format.Both outputs can be tweaked with respect to time intervals, granularity of the data and specific slice that a user wants to monitor.While the second output layout is intended for visualization purposes, it provides useful data when used for specific time intervals or occasions.As such, to supply a more continuous visualization Regarding the network slices implementation, the usage of Software Defined Networks was crucial.Outside the 5G Core, each slice is represented by a Virtual Local Access Network (VLAN) configured in the virtual infrastucture and the switch.Even though the VNFs are deployed via an orchestrator in OpenStack, each is connected to a unique VLAN, maintaining the end-to-end character.To ensure isolation, the OpenFlow configuration of the SDN Switch ensures that no traffic is routed between VLANs, despite the Openstack configuration.For this to happen, packets originated in each slice are marked accordingly with Differentiated Services Code Point (DSCP) values.In order to conclude whether the traffic was routed to different slices or not, a relation between a DSCP value and VLAN/Slice is established previously, in order to trace packets that were routed between VLANs.With this implementation, it is safe to say that the solution provides a good scalability factor, since each slice only requires a creation the configuration of a new VLAN.

VI. EXPERIMENTATION AND RESULTS
To test the monitoring solution, it was required to generate network traffic in the infrastructure.Thus, and in the context of the 5GASP project (H2020-ICT-2020), multiple Virtual Network Functions, packaged inside a single Network Application, were deployed in an Openstack9 cluster via OSM 10 .Said cluster was implemented with internet connection and connectivity to an on-premise 5G Core, as seen in Figure 5.
This scenario mimics the deployment scenario of the 5GASP project, where Network Applications are deployed in an Openstack cluster, connected to the 5G network through the described SDN Switch.In the scope of the 5GASP project, this SDN switch already enabled the 5GASP Network Applications to communicate through the 5G Network.However, with Fig. 6.Impact of the traffic load on CPU performance the work we now present, we evaluate its effectiveness in generating monitoring metrics that will allow us to conclude on the network consumption of the Network Applications.
Two test scenarios were defined, aiming to conclude on the performance of the solution and also the accuracy of the data retrieved, respectively.The first scenario intended to analyse the performance drop of the SDN Equipment as well as the implemented Exporter service when dealing with network overload.To analyse the performance drop of the SDN equipment, we relied on evaluating the CPU load variation.It is known that high CPU loads affect the processing and transmission time of incoming packets.Furthermore, it may also affect the switch's ability to enforce QoS policies or perform traffic management functions.Due to this, we analysed if the CPU load was heavily increased when an increased amount of traffic was being processed by the SDN equipment.iPerf3 11 was the main tool used for both performance and accuracy, using virtual machines and a UE connected to the 5G Network, respectively.Concerning the performance of the solution, the tests performed generated increasing amounts of traffic from 100 Mbps to 10 Gbps.The virtual machines consisted of 2 Ubuntu 22.04 images deployed in an Openstack cluster, both equiped with 4 GBs of RAM, 2 CPUs and 40 GBs of storage, generating 0.2, 0.5, 1, 5 and 10 Gbps of traffic.Each VNF was deployed in an isolated VLAN which was also configured in the SDN Switch, forcing it to forward the traffic between the virtual machines.The switch showed little to no change in processing load, as shown in Figure 6.
The consistency of these values is a result from the high switching capacity of the equipment integrated in the testing scenario, roughly 256 Gbps.Thus, and bearing in mind the limitation of 10 Gbps when generating traffic in the infrastructure, the performance of the switch was not affected.As for the Metrics Exporter and Processor, it is concluded that the network overload induced no significant changes in 11 https://iperf.fr/Fig. 7. Comparison between real and measured throughput value performance.This is due to the fact that the data retrieval process via SDN Controller is periodical and stable.The statistics are scraped scattered through time and the amount of data retrieved is consistent, not depending on the amount of traffic forwarded via the switch.
Regarding the second testing scenario, the accuracy of the solution was tested in two different ways.The first was by generating a constant, pre-defined throughput value to later query in the visualization platform or via the Metrics Exporter and Processor.The second, although similar to the first one, only relied on the RTT values measured between a UE and the Network Application, mimicked through a VNFs.
As a UE, the equipment used was an Intel NUC 11 with 16 GB of RAM and 250 GB of storage, connected to the 5G Network via a Sierra Wireless EM9191 5G Modem and a Techship MU201 Adapter.Regarding the first test, the UE generated traffic via iPerf3, targetting a VNF hosted on OpenStack, deployed in a VLAN matching the slice to which the UE was connected.The generation of network load was done in 3 experiments, targeting different throughput values.Each experiment consisted of 5 runs of 20 second intervals producing network traffic, and the results, shown in Figure 7, were obtained from the average of all runs.The metric gathering process was efficient due to its quick reaction to fluctuations in network traffic volume.As for the precision of the values, the results prove that the solution offers accurate throughput values.Regarding the second test, we conducted 3 experiments, comprised of 20 runs.In each run, the RTT values were measured for 20 seconds.The average results are showcased in Figure 8, and, once again, point out that the metrics provided by our solution are acceptable.
Our results show a minimum error of 3.36% and a maximum error of 7.25%, when considering the tests related to the network's available throughput.Regarding the results of the tests that considered the RTT values, the minimum error was of 2.56% and the maximum error was of 5.98%.Given that our solution comprises different components for data gathering, conversion, and exporting, the obtained error margin is easily understandable.The communication between these components and the fact that we rely on third-party software to convert traffic data into metrics can easily impact the accuracy of the values obtained.

VII. CONCLUSION
Network monitoring can be significantly beneficial in resolving issues for a variety of reasons.These include early detection of service failures, a proactive approach to these failures, quick pinpointing of a system's malfunctioning, among others.When dealing with 5G Networks, it's particularly relevant to monitor network traffic, since many components are software-based and virtualized, and service issues may be harder to detect due to the underlying infrastructure.However, due do the recent and ongoing development of 5G and SDN products, traffic data is not yet exposed for third party usage.The solution proposed in this paper was developed to overcome said difficulties while also using state-of-theart technology.After its incorporation in the 5GASP project and diverse performance tests, it is safe to say that the main goal was achieved.All VNF network traffic was measured effectively, both in 5G Network and internet access, making it possible to assess their impact on the infrastructure and preventing future failures or noncompliance with predefined standards.

VIII. FUTURE WORK
While the developed solution achieves its main goal of measuring VNF impact on a network, its features could be enhanced to provide a wider variety of information.Despite the significance of throughput to evaluate the network performance, it would be beneficial to retrieve other metrics such as jitter, or packet loss.Said indicators could provide better insights not only on the network's own functioning but also on the VNFs network performance and possible errors in the deployment process.
Moreover, a more assembled monitoring platform would be useful in order to achieve more accurate measurements.The fact that many components had to be implemented to gather, convert, and export the metrics induced inevitable delay in the transmission of the data, which makes the overall solution less agile.The usage of third party software to convert traffic data into metrics can also influence the accuracy of the values.The process of retrieving data from the switch using an SDN Controller is more prone to faulty values due to the delay in the communication between the two.Finally, the conversion of statistics into time based metrics also increases the risk of achieving slightly different values.Since the gathering service is responsible for the periodical data retrievals, network and processing delays in said service's host can also influence the end result of the metrics exporting process.

Fig. 8 .
Fig. 8.Comparison between real and measured RTT values