Reliability of UAV Connectivity in Dual-MNO Networks: A Performance Measurement Campaign

Unmanned aerial vehicles (UAVs) can be controlled in beyond-visual-line-of-sight use cases via today's commercial mobile network operator (MNO) deployments, such as those based on long-term evolution (LTE) technology. In order to improve the reliability of UAV connectivity, especially in more critical use cases, several MNO networks may be utilized simultaneously. To evaluate this option in typical environments, performance measurements were conducted over two public LTE networks in urban, suburban, and rural areas in/near the city of Tampere, Finland, and at various UAV flight altitudes: 1 m, 15-20 m, and 50 m above the ground. The results indicate that there are distinct benefits in utilizing more than one MNO for improved levels of connection reliability.


I. INTRODUCTION
The utilization of unmanned aerial vehicles (UAVs), also commonly known as drones, is reaching the point where beyond-visual-line-of-sight (BVLOS) operations are becoming increasingly more feasible. In the U.S. alone, it has been reported that thousands of BVLOS operations have been conducted over the recent years. By utilizing BVLOS applications of drones, it was estimated that, for instance, power line inspection efficiency can improve two to three times as compared to visual-line-of-sight (VLOS) operations over just one day [1].
Typically, this also means that UAV control is performed via public networks deployed by mobile network operators (MNOs), such as today's long-term evolution (LTE) systems. In these cases, connectivity can be categorized into droneto-infrastructure (D2I) or infrastructure-to-drone (I2D) options. Another alternative is to utilize flying ad hoc networks (FANETs) wherein communication is established through other nodes in the device-to-device (D2D) regime as studied in [2], [3], and [4].
However, due to open challenges pertaining to longer latencies in FANETs, they may not be suitable for reliable BVLOS operations. Conventional point-to-point control links between the drone and its remote control (pilot) have a limited range of up to a couple of kilometers in distance. On the other hand, mobile networks are capable of providing nationwide coverage, which makes them particularly suited for the BVLOS use cases to enable longer communication distances between the drone operator and the UAV.
Research on how well contemporary mobile networks can support drone-centric connectivity in different environments has already been attempted in several publications after the initial studies on supporting flying user equipment (UE) by the 3rd Generation Partnership Project (3GPP). The latter were summarized in technical report (TR) 36.777 with the focus on LTE networks. A crucial aspect of utilizing LTE for drones in BVLOS operations was addressed in [5], where the authors noted that the command and control availability can be satisfied when only a few aerial UEs operate in the same region.
Combining simulations and field measurements, the authors in [6], [7], and [8] compared the basic key performance indicators (KPIs) for flying UEs and regular ground UEs. As one would expect, the results showed that the probability of having a line-of-sight (LOS) connection between a UE and its serving base station antenna increases with height. Therefore, bringing the UEs to higher altitudes above the ground may improve the received signal power levels. However, at the same time, interference can grow as neighboring cells also become more susceptible to having a LOS exposure to the UE.
The authors in [9] demonstrated how drones carrying UEs can be separated from the ground UEs, which is important for developing cellular network functionality for flying UEs, since terrestrial networks are currently optimized for typical groundlevel usage. There has also been a concern that airborne UEs may cause additional interference to regular mobile network users. However, the authors in [10] concluded that operating a small number of drones does not have a critical impact on the ground-level UE performance, albeit it degrades the efficiency of radio resource utilization in terms of higher transmit powers and more resource blocks.
In stringent use cases, the need for having more than one communication link from a drone to the network can become beneficial, especially to support the emerging features like ultra-reliable low-latency communication (URLLC) in fifthgeneration (5G) cellular systems. The authors in [11] and [12] concluded that URLLC capabilities could be regarded as a promising option for enabling the BVLOS operations.
This paper addresses important performance aspects of using mobile networks for drone communication in different environments and at various heights above the ground. A particular emphasis is set on the comparison of utilizing either one public mobile network or two of them at the same time. This includes system design wherein the command-andcontrol messages are sent through both networks simultaneously and the first packets to arrive from any network are utilized for the actual drone operations. Another option is to employ one of the networks first and only switch to using the other one should the connection have problems, with an expense of higher delay for control operations. Therefore, this work aims to compare two simultaneous connections while their specific applications are left open. For example, the use of carrier aggregation or beamforming techniques may improve the radio access performance of a single network; however, the system will be required to support those features.
A comparison between two MNOs performed in this study targets the main KPIs, with an essential goal to evaluate communication reliability, which is one of the open problems for UAV-based connectivity as highlighted by the authors in [13]. The utilization of dual-MNO networks can mitigate potential connectivity problems that might occur in the case of resorting to a single mobile network. Therefore, it is topical to assess the benefits of employing more than one MNO deployment for delivering radio connectivity to UAV-centric services. These results become of a major benefit for the entire community to determine the suitability of utilizing mobile networks as the backhaul carrier for connected drones.

II. SCENARIOS OF INTEREST AND SYSTEM CONFIGURATION
In this study, measurements were conducted over two public mobile networks deployed by different MNOs employing the frequency division duplex (FDD) LTE technology, which has been suggested as a promising candidate for initial deployments of small UAVs in [14], [15], and [16]. Hence, in all of the considered cases, two networks were assessed simultaneously. Furthermore, the measurements targeted three dissimilar environments: urban, suburban, and rural. In all of these areas, three distinct altitudes were selected with one being at the ground level, which is the regular use case for mobile systems.
The highest altitude constituted the maximum allowed flying height of 50 meters above the ground, which is due to the close proximity of the airport in Tampere  The measurements were performed with a commercial drone, DJI Inspire 2, which had two Samsung Galaxy S8 phones mounted on it, such that two different measurement software tools could be utilized: one phone had Keysight Nemo Outdoor (NEMO) and the other phone had Ro-hde&Schwarz QualiPoc (QPOC). These software modules did not prefer connections to any technology, so that the phones could utilize the networks as regular UEs and connect on different frequency bands throughout the measurements. The devices were attached to the left and the right sides of the drone (see Fig. 1), while the measurement routes were automated and traveled in total four times per flying altitude and route according to the configurations in Table I. With these four configurations, at least one operator measurement per drone side per measurement software tool was recorded. The ground level measurements were conducted with a bicycle carrying the phones at the height of 1 meter above the ground level in urban and suburban cases. In the rural scenario, the ground level measurements were taken with the help of a car, by keeping the measuring phones outside of it in order to avoid additional losses from the vehicle's body. The utilization of a bicycle and a car for the ground-level measurements was preferred over flying a drone too close to the surface for the purposes of safety. Fig. 2 shows our different setups for the measurements with a drone, a bicycle, and a car.
3GPP's Release 15 studies offer the data rate requirement of 60-100 kbps for the command-and-control traffic. Therefore, phone measurement software was configured to continuously  communicate a small amount of data. The latter was adjusted to the packet size of 1000 bytes; while the time between packets in both directions was fixed to four seconds in order to not have the connected mode at all times and study the successful connection establishment. This resulted in having eight seconds between the two consecutive downlink or uplink messages, thus translating into the average data rate of 125 kbps. All of the measurement routes are displayed in Fig. 3. These were pre-planned with a commercial Litchi software to enable automated and repeatable flight paths. The time it took to fly with a single configuration in each location varied from 2 minutes to 3 minutes with about 400 m to 600 m flight lengths having the drone flight speed of 18 km/h. The data collected during each flight was constrained to the automated flight route path; therefore, the measurement data from takeoff until the start of the automated flight and after the end of the route until landing has been excluded from the results.

III. ESSENTIAL PERFORMANCE RESULTS
The measurements collected with the utilized equipment are discussed separately for each environment in what follows. The reference signal received power (RSRP) and signal-tonoise ratio (SNR) values are collected in Table II and Table III for all the cases to assess the channels from the MNO viewpoint in these different scenarios. They are produced by combining the exported data from all the configurations in Table I with the aid of Matlab.
The operators are differentiated by using a distinct background color in the tables. The figures for the rural area are  not displayed as in these cases the measurements for RSRP and SNR values were unavailable for the other operator as universal mobile telecommunications system (UMTS) connectivity was utilized due to poor LTE network quality in the region. Therefore, SNR was not reported by the measurement software as the latter was assessing only UMTS network performance and did not include the SNR values. The same effect occurred for the MNO A at the ground level and for the MNO B at the 50 m height above the ground in the suburban environment. The rate of negative acknowledgements (NACKs) for the hybrid automatic-repeat-request (HARQ) protocol was used as the baseline for evaluating the reliability of the networks. For LTE, this is straightforward as it characterizes the handling of erroneous packets on the physical layer. However, this is different for UMTS as there is also a variable named statistical discontinuous reception, which is reported together with the acknowledgement (ACK) and NACK messages. This results in not having comparable parameters between these two mobile network technologies. Therefore, in the cases where UMTS was preferred over LTE by the system, the output was not shown for the UMTS data to avoid confusion.

A. Urban Environment
The urban scenario was assessed in Tampere, Finland, in the Tampella district, during summer 2019. The results presented in Fig. 4 display the proportion of NACK messages in the urban environment. It can be observed that the two operators   perform differently and that one of them outperforms the other one distinctly. Therefore, by utilizing two MNO networks simultaneously, the overall connection reliability levels can be improved as compared to the use of only one public LTE network for the connectivity purposes.

B. Suburban Environment
Similar to the urban use case, Fig. 5 demonstrates the results of NACK message drops for both operators at different altitudes above the ground in the suburban area. The measurements were collected in Hervanta district of Tampere, Finland, in summer 2019. Both MNOs experienced certain challenges in their LTE network connectivity, which caused a downgrade to UMTS, or 3G mobile network, during the measurements.
The data is omitted for the cases without the LTE service and marked as not available (n.a.) in the legends of the figures. Interestingly, one operator observed difficulties at the highest altitude, whereas the other one faced challenges on the ground level. This indicates that the neighboring base station antennas might have had different tilting and orientation in the measurement area; hence, there are distinct dissimilarities between the two MNO deployments.

C. Rural Environment
The rural measurement area was located south of Tampere, in the region of Suonijärvi, in Lempäälä, Finland (also summer 2019). In this case, one of the MNOs had insufficient LTE coverage across the entire area, which caused it not being able to utilize LTE at all. Therefore, the results in Fig. 6 are not showing the said operator since NACK message performance in the UMTS network is not directly comparable with that in the LTE deployment.

IV. CONCLUSIONS
In this work, communication reliability when utilizing public mobile networks for UAV command-and-control traffic was studied with a drone having cellular access via two public LTE networks in Finland. The command-and-control messages were generated and sent in both uplink and downlink with the data rate set according to the 3GPP requirements. The reliability was then assessed by monitoring the NACK messages to indicate the proportion of packets that were lost during such operation.
The findings suggest that not only there is a difference between the UAV flight altitudes above the ground, but also there are distinct performance variations for the two MNO deployments across the same locations at the same time, as well as with respect to various environments and altitudes. This accentuates the impact of the base station antenna orientation (with the sidelobes and nulls between them) especially in the vertical direction, together with the absolute distance to the UAV from those antennas. As there is diversity between the considered networks in all cases, there should be a room for reliability improvements by maintaining more than one MNO service simultaneously.
Our results also indicate that the conventional assumption of seeing higher received signal strength with growing altitude above the ground remains somewhat accurate outside the city, but within the city limits, there were cases when this did not hold. From the results presented in Table II and Table III, it can be concluded that the strongest signal levels in urban areas were recorded on the ground level, and not at higher altitudes as one would expect. This is in-line with the UMi features indicating that most of the signal power should concentrate at the below 10 m heights above the ground.
The same can be noted for the suburban areas, but there the highest signal strength is observed at the 20 m height. This reflects well the UMa deployment and suggests that this environment has the average antenna height closer to 25 m rather than to 10 m in UMi or to 35 m in RMa cases. Moreover, the highest signal power levels in the rural areas are those recorded at the 50 m altitude above the ground, which reflects the RMa deployment considerations. However, the SNR behavior as a function of the altitude above the ground continued to meet the expectation of observing lower values at higher altitudes due to increased overall interference levels. This learning can be useful for both of the involved operators and confirms that the chosen measurement locations are within the ITU-R guidelines.
The outcomes of this study call for extended investigations that target to compare in more detail the differences of utilizing various mobile network technologies for both D2I and I2D connections in dissimilar environments and underlying mobile operator deployments. These may include new commercial 5G layouts as they will have acceptable service coverage whenever URLLC capabilities will also become available. Moreover, suitable performance indicators need to be proposed to enable a fair comparison between different mobile network generations (from second to fifth generation) for assessing the reliability of UAV connectivity across all of these cellular technologies.