A Quantitatively Derived NMAC Analog for Smaller Unmanned Aircraft Systems Based on Unmitigated Collision Risk

The capability to avoid other air traffic is a fundamental component of the layered conflict management system to ensure safe and efficient operations in the National Airspace System. The evaluation of systems designed to mitigate the risk of midair collisions of manned aircraft are based on large-scale modeling and simulation efforts and a quantitative volume defined as a near midair collision (NMAC). Since midair collisions are difficult to observe in simulation and are inherently rare events, basing evaluations on NMAC enables a more robust statistical analysis. However, an NMAC and its underlying assumptions for assessing close encounters with manned aircraft do not adequately consider the different characteristics of smaller UASonly encounters. The primary contribution of this paper is to explore quantitative criteria to use when simulating two or more smaller UASs in sufficiently close proximity that a midair collision might reasonably occur and without any mitigations to reduce the likelihood of a midair collision. The criteria assumes a historically motivated upper bound for the collision likelihood and subsequently identify the smallest possible NMAC analogs. We also demonstrate the NMAC analogs can be used to support modeling and simulation activities.


II. Introduction
The continuing integration of unmanned aircraft system (UAS) operations into the National Airspace System (NAS) requires new or updated regulations, policies, and technologies to maintain safe and efficient use of the airspace. To help achieve this, regulatory organizations such as the Federal Aviation Administration (FAA) and the International Civil Aviation Organization (ICAO) mandate the use of collision avoidance systems to minimize the risk of a midair collision (MAC) between most aircraft. These systems are often defined by a set of performance requirements. Over the past decade, these requirements have been adapted to reduce the risk of MACs between manned and unmanned aircraft [1,2,3,4].
As part of the development and evaluation of collision avoidance systems, the quantitative near midair collision (NMAC) definition of a simultaneous loss of 500 feet of horizontal separation and 100 feet of vertical separation is used for safety modeling and simulation. The quantitative NMAC definition also differs slightly from a policy-based definition of a loss of 500 feet of separation that is intended for incident reporting, investigation of dangerous encounters, and characterizing trends of airborne risk. In all of these contexts, NMAC acts as a surrogate for a MAC either due to the rarity of MAC or as a quantitative measure of likely MACs. Since NMACs occur more frequently, this enables more robust statistical analysis; MAC statistics are difficult to calculate directly when using point mass aircraft models. Historically, the aviation community generally assumes manned aircraft experience an NMAC at least ten times more often than a MAC [5].

A. Motivation
One of the most important metrics for safety assessments is the rate at which an NMAC occurs over many simulated encounters. An NMAC is considered a hazardous state, and it should be avoided to minimize a catastrophic incident (e.g. MAC). This metric has been used to evaluate and determine the risk of a MAC for close encounters, where the threat of a MAC exists. Historically, close encounters have been defined as when a MAC is likely within 15-60 seconds. This time horizon was first proposed as a means to evaluate manned aircraft avoidance maneuvers [6] in the 1950s, implemented as part of the algorithmic design for the first mandated collision avoidance system for manned aircraft in the 1980s (TCAS) [7], and further adopted for characterizing encounters between small UAS and manned aircraft [2] in the 2010s. Given that NMAC is a hazardous state that warrants a collision avoidance maneuver, well clear is characterized as a quantitative state that implies a collision avoidance action will likely be unnecessary [2]. Thus often longer encounters to evaluate well clear performance are also assessed using NMAC.
Quantitative collision avoidance and well clear criteria not only act as design targets, but also act as "measuring sticks" for assessing the goodness of a given UAS-specific detect and avoid (DAA) system [1]. DAA systems help enable beyond-visual-line-of-sight (BVLoS) operations by reducing the risk of a UAS being a collision hazard within the NAS. Fast-time Monte Carlo simulations are often utilized to evaluate DAA systems by estimating the probability of an NMAC. Simulated encounters are generated based on what DAA function is being evaluated and the performance or behavior of the aircraft involved. For example, encounters meant to prompt collision avoidance maneuvers may not be suitable to evaluate a DAA system attempting to maintain well clear.
Encounters with strictly smaller UAS participants are typically different from those with manned participants. UAS encounter each other at altitudes and airspeeds dissimilar to those exhibited by manned aircraft. For example, the current model representing manned aircraft that squawk a discrete code has an initial altitude floor of 1000 feet above ground level (AGL) [8]. Many smaller UAS-only encounters, however, are expected to occur at lower altitudes.
Additionally, UASs are generally significantly smaller than manned aircraft, with many low-altitude UASs having a wingspan of 15 feet or less.
The current criteria and metrics for assessing close encounters with manned aircraft do not adequately consider the different characteristics of lower altitude smaller UAS-only encounters. While NMAC could be used to evaluate UAS to UAS encounters, we assert that the NMAC volume is too large as it over estimates the MAC risk and isn't appropriate for operational suitability evaluations. In response for modeling and simulation-based evaluations, a new discrete quantitative definition is required to assess if two or more UASs were sufficiently close enough that a MAC could reasonably occur. Subsequently, this poses questions related to safe and efficient integration of smaller lower altitude UASs into the NAS, such as: • What is the separation assurance criteria, if any, for UAS-only encounters?
• What models of aircraft behavior and operational concepts are required for UAS-only encounters?
• What is the likelihood of an UAS encountering another UAS, given specific operational concepts?
• What is the maximum permissible latency between different collision avoidance functions?

B. Scope
This work considers only unmitigated quantitative assessments of close airborne encounters between smaller low altitude UASs and is intended to be compatible with existing safety frameworks. Non-airborne risk assessments, such as ground strike, durability, or reliability are out of scope. The scope of this work was informed by the needs of the FAA TCAS Program Management Office and UAS Integration Office and, along with the activities of the standards development organizations of ASTM F38, RTCA SC-147, and RTCA SC-228. Initial scoping discussions were also informed by the UAS ExCom Science and Research Panel (SARP), an organization chartered under the ExCom Senior Steering Group and the SARP's previous research on quantitative separation criteria for encounters UAS and manned aircraft [1,2,3]. However, the SARP did not provide a final review of the research.
The scope consisted of commercial, civil, or military smaller UAS operations in the United States airspace with the exception of Classes A and E-above-A. Recreational or amateur UAS operations governed by 14 CFR Part 101, 49 U.S.C. 44809, and AC 91-57B were out of scope. The scope was limited to smaller UAS with wingspans of 25 feet or less, but there were no restrictions on airspeed or vertical rates. This wingspan limit was originally recommended by Lester and Weinert [1] and assumes that manned general aviation and commercial transport aircraft have wingspans greater than 25 feet. The scope of the concept of use for the Airborne Collision Avoidance System (ACAS) Xu for Smaller UAS (sXu) (dated February 28), RTCA SC-147 ACAS sXu terms of reference, and ASTM F3442/F3442M -20 DAA performance standard 6 include a similar wingspan limit. Additionally, all small UAS weighing less than 55 pounds maximum gross take-off weight (MGTOW) are also smaller UAS. Note that not all smaller UAS weight less than 55 pound MGTOW. Any conclusions or interpretations of this paper could be applied to just small UAS defined by MGTOW.

C. Objectives and Contributions
We focused on two objectives identified by the aviation community to support integration of UAS into the NAS. First to quantitatively define a close encounter between UASs. The other to provide criteria for a quantitative metric to support safety modeling and simulation efforts, especially when using point-mass aircraft models. These contributions are intended to support current and expected UAS DAA system development and evalu ation and facilitate stakeholder engagement to refine our contributions for policy-related activities, such as pilot reporting.
The primary contribution is an exploration of criteria for the quantitative metric based on unmitigated airborne collision risk and the historical assumption about a MAC likelihood, with three of the smallest candidates proposed by the authors. An additional contribution was smaller UAS-only encounter sets that where simulated to support and demonstrate development of detect and avoid capabilities. The candidates' applicability was limited based solely on wingspan and height. This new criterion is intended to be functionally analogous to NMAC, which has been used in safety modeling and simulation efforts of encounters with manned aircraft and large UAS for several decades. To its credit, near mid-air collision is a term that expresses exactly what it is with clarity. To continue this tradition of clear terminology, the new criteria is called a smaller near midair collision (sNMAC).
A concurrent effort demonstrated that the authors proposed sNMAC candidates, informed by unmitigated risk, can be used to evaluate how a DAA system mitigates the risk of a MAC [9]. This paper focus on the establishing the rational for sNMAC, an analysis of smaller UAS, a quantitative exploration of unmitigated risk, and then overviews that close encounters suitable for a sNMAC-based evaluation can be reasonable generated. Refer to the concurrent publication for details on the mitigated analysis using the sNMAC candidates.

III. Literature Review
In this section we review the history of characterizing and defining NMAC for manned aircraft operations. While the scope of our work was limited to quantitative assessments, the use of the NMAC terminology has been widespread for decades in additional contexts. In response, both the policy and quantitative perspectives were reviewed to provide historical context of the MAC problem and motivate the use of a quantitative NMAC to support the development of collision avoidance systems.

A. History of the NMAC Terminology
The use of the NMAC terminology predates the creation of the FAA. As of 1955, a standard quantitative NMAC had not been established [10]. Roessger [11] in 1958 noted that statistical samples of qualitative NMACs have been collected since 1948 and used due to the limited occurrences of MACs. A decade after the FAA was formed, the Near Midair Collision Study Program of 1968 (AC 00-23) and its follow-up NMAC study group were tasked to develop reliable information and guidance to reduce hazards associated with NMACs [12]. The study group quantitatively defined NMAC as a hazardous situation with a separation of less than 500 feet (no individual horizontal and vertical components) and qualitatively assessed the severity as either critical or potential: 1) Critical-A situation where collision avoidance was due to chance rather than the act on the part of the pilot 2) Potential-An incident which might have resulted in collision if no action had been taken by either pilot A decade later in 1979, the FAA Task Force on Aircraft Separation Assurance [13] largely adopted the preceding terminology, but refined the critical severity to include a reduction in the loss of separation to 100 feet. The critical severity continued to assume that human pilots had no means to influence the likelihood of a MAC, given an NMAC.
In the subsequent decade, the 1989 "Report of the Interagency Near Midair Collision Working Group [14]," along with FAA Order 8020.11 and NASA Aviation Safety Reporting System's "Standard Operating Procedures Manual No. 2" provided no significant changes to the preceding NMAC terminology for policy and reporting. A minor emphasis was that a NMAC was an incident that can involve more than two aircraft and defined as a singular incident rather than pairwise aircraft states.
The current use of NMAC for policy aligns with the historical precedent. The hazard severity associated with a NMAC is currently codified in FAA Order 8020.11D 7 and policy related to NMAC investigations are described in the FAA Flight Standards Information Management System (FSIMS) 8900.1 8 .

B. Use of NMAC to Support Development of Collision Avoidance Systems
Interest in an onboard collision avoidance capability was expressed at the formation of the FAA after the 1956 Grand Canyon MAC. However, decades passed until the community adopted a consistent quantitative NMAC definition. Alexander [15] in 1970, when describing the Department of Air Traffic Control Advisory Committee, defined NMAC as a state where separation is less than 250 feet. A few years later in 1973, Jones and Lutze [16] numerically assessed in simulation a range of separation criteria from 0.05 nautical miles (≈ 300 feet) to 0.25 nautical miles (≈ 1500 feet). In 1971, Hulsman [17] qualitatively defined NMAC as "…if one or both of the pilots involved found it necessary to take immediate evasive action in order to prevent a collision," or as "…if two aircraft come in very close proximity to each other but not so close to have caused a collision if no evasive action was taken." Newton [18] in 1978, when assessing a specific restricted airspace, also did not provide a quantitative definition for NMAC, instead defining it as "…having the potential of a considerable loss of life." The advancement of transponders and beacon-based technologies in the 1970s enabled the prototyping of the Beacon Collision Avoidance System (BCAS) for operations in low-density airspace [19]. In BCAS technical reports by Clark and McFarland [20] and Grupe et al. [21], NMAC was not explicitly defined; rather, they specified that a collision threat was defined by both relative range (horizontal) and relative altitude criteria. Furthermore, in another BCAS report, Welch and Orlando [22] when describing the BCAS avionics, declared that "Mode C is used because collision avoidance logic requires measurement of range and altitude." Multiple MACs in the late 1970s [23,24,25], further spurred development of BCAS. TCAS was developed using the fundamental concepts of BCAS but enhancements were made to enable operations in high -density airspace [19]. In December 1983 Lebron et al. [7] authored "System Safety Study of Minimum TCAS II," which assesses the overall 7 http://www.faa.gov/documentLibrary/media/Order/FAA_Order_8020.11D.pdf 8 http://fsims.faa.gov/PICDetail.aspx?docId=8900.1,Vol.7,Ch4,Sec1 safety characteristics of the use of TCAS. They cited FAA policy regarding varying NMAC sensitivities and chose to use the critical severity. Additionally, based on our discussions with those with first-hand knowledge of initial TCAS development, the physical dimensions of the aircraft intended to be equipped with TCAS were one of the main considerations when quantitatively defining NMAC. A minor consideration was altimetry quantization for rounding to an order of magnitude. Thus, Lebron et al. defined NMAC quantitatively as a cylindrical boundary such that "…at the closet point of approach, the vertical separation is less than 100 feet and the horizontal separation is less than 500 feet [7]." Prototyping TCAS for helicopters began in earnest after the 1983 report by Lebron et al. To support development, Taylor and Adams [26] in 1985 reported the results of a survey examining helicopter and pilot behavior and preferences. They did not use the criteria defined by Lebron et al., instead defining NMAC as an "…incident in which an intruding aircraft approached within 1000 feet, and which required an evasive maneuver to avoid a collision or in which no opportunity for evasive action was available [26]." However, like Lebron et al., they did not vary the NMAC definition based on proximity to airports, traffic density, and operating altitudes. In spite of different separation criteria, they concluded that the "NMAC environment of rotorcraft does not differ significantly from that of light general aviation aircraft [26]." They recognized the significance of operating at low altitudes, which will influence potential avoidance maneuvers and the risk of maneuvering into obstacles or terrain.
Proceeding reports on TCAS safety assessments and developments by Spencer in 1989 [27] and McLaughlin and Zeitlin [28] in 1992 cited Lebron et al. [7] when defining NMAC. Notably, like previous reports, they referred to NMAC with a critical severity, using language such as "critical NMAC event." McLaughlin and Zeitlin also emphasize how position error affects collision avoidance performance while leveraging a consistent NMAC definition.
The FAA's development of TCAS's successor, the Airborne Collision Avoidance System X (ACAS X), prompted a re-evaluation of the separation criteria and metrics used for collision avoidance. The manned variant, ACAS Xa, intended to replace TCAS on commercial aircraft, bases its safety metrics [53] on the same NMAC definition used for BCAS and TCAS. ACAS Xu, designed to meet RTCA DO-365's DAA requirements, also uses this NMAC definition. ACAS sXu [31], a variant that provides collision avoidance for smaller UAS, however, requires a quantitative definition that is appropriately tailored for operations by these new airspace entrants.

IV. Method
In this section we review the unmitigated risk-based method used to produce the potential candidate sNMAC criteria. The underlying methodology was primarily based on unmitigated collision risk but was informed by operational suitability including an assessment of impact on a candidate DAA system. It is similar to previous efforts to define quantitative separation criteria for encounters between UAS and manned aircraft [1,2,3] and has been vetted through standards-developing organizations. The method was also not framed as a validation exercise where an initial candidate was proposed and endorsed; rather the method relies on clearly defining a set of assumptions and quantitatively evaluating a set of candidates. Specifically, the method can be applied as an iterative process and consists of the following: 1) Define candidate sNMAC forms and risk thresholds 2) Define models and distributions, given appropriate assumptions 3) Evaluate and demonstrate candidates via unmitigated Monte Carlo simulations, including a sensitivity analysis 4) Forum with subject matter experts (SMEs) to downselect candidates based on assembled metrics 5) Socialize the process and results with stakeholders and transfer supporting material to community

A. Threshold and Likelihood of a Midair Collision
The probability of a MAC is partly influenced by the sum of wingspans, but historically for manned aircraft, there has been an assumption that there are 1 MAC for every 10 or more NMACs [5]. However, Kochenderfer et al. [5] empirically assessed that the probability of a MAC, given an NMAC, was rarer and likely less than 10%. To be consistent with the historical expectations associated with an NMAC, the sNMAC candidates were evaluated based on the likelihood of a MAC. For encounters between any types of aircraft, MACs should continue to be rare events.
We assumed that a MAC should occur no more than approximately once for every 10 sNMACs. We assumed a MAC rate based on the assumption that the aviation community historically accepted rather than the observed or simulated rate. We also assumed that the community would not accepted a MAC rate greater than 10% but could potentially accept a more stringent MAC rate less than 10%. Given these assumptions, our results are assumed to the smallest possible sNMAC criteria that the aviation community should accept. If a more strigent MAC rate was preferred, the spatial volume of sNMAC would need to increase.
There is a tradeoff that must be satisfied, where we assumed UAS operators prefer sNMAC be as small as practical while maintaining an acceptable level of MAC risk. In general, UAS operators desire separation standards that minimize the impact to their operations. The acceptable MAC rate was a SME decision; this is subject to validation and should be a focus of future work.

B. Form and Shape
The literature review provided no technical justification for why NMAC was designed as a cylinder with a 5:1 ratio between the horizontal and vertical dimensions. Instead, BCAS and TCAS technical reports asserted the need to identify the threat of a MAC based on range and relative vertical distances. To align with the BCAS, TCAS, and ACAS X efforts, potential sNMAC candidates were quantitatively defined by a loss of horizontal and vertical separation. The ratio between the dimensions was not specifically considered, and each dimension was considered independently. The minimum dimensions of the volume should also not be less than the sum of the effective widths or heights; two smaller UAS should be able to "fit" within a cylinder of the minimum dimensions. To be accessible and align with aviation norms, dimensions were multiples of 5, such as 25 feet.
The form should not change depending upon altitude, aircraft performance, location, operating limitations, time, or wake turbulence. When assessing safety risk associated with manned aircraft operations, NMAC is not the only outcome considered. Safety cases may utilize other parameters or other variables, such as the number of encounters with other aircraft. Exposure is usually characterized per flight hour. By comparison, the NMAC definition is the same for an encounter between two general aviation aircraft flying 100 knots each at 3000 feet and an encounter between two large transport cargo aircraft at 30,000 feet: the NMAC definition is independent of 14 CFR 91 and related regulations. Also, despite most manned operations occurring at higher altitudes, most NMACs occur at lower altitudes. Therefore, altering or scaling the sNMAC volume based on altitude lacks precedent and adds complications without discernible benefit. In response, the operating limits defined by 14 CFR 107.51 were not considered.

C. Assumptions
While the distribution of UAS wingspans and heights, as discussed in the following section, are the primary factors when defining and evaluating candidates, there are a variety of secondary safety and operational considerations.

Position Error and Observability
The observability of and determination that a sNMAC occurred due to position error was not a consideration. Specifically, altimetry quantization, ADS-B Navigation Integrity Category (NIC), or ADS-B Navigation Accuracy Category -Position (NACp) had no influence on selecting sNMAC candidates. These errors and uncertainty are important when evaluating safety. These uncertainties and errors should also be applied to calculate the distribution of risk given an encounter with a sNMAC.
Incorporating sensor errors and uncertainty is in alignment with current uses of the NMAC definition and helps mitigates the risk of false positives and negatives. The accuracy and resolution of range and altitude measurements should be characterized to support simulations using sNMAC or help define performance requirements for DAA systems that minimize the sNMAC and MAC threat.
Sensor errors and uncertainties can be considered when evaluating safety. sNMAC does not necessarily need to be so large as to contain all uncertainties. A historical analogy is that NMAC was quantitatively defined to assess TCAS, and aircraft equipped with TCAS must me et the minimum Mode C quantization requirements set by 14 CFR 91.36. This 1983 standard requires an "indicated altitude to within ± 125 ft. on a 95 percent (two sigma) probability basis [7] …"

Aircraft and Pilot Performance
As a singular metric, NMAC does not vary based on aircraft or performance of the pilot in command. We make the same assumption for smaller UAS and did not vary sNMAC based on advertised speed, vertical rate, turn rate, or other aircraft performance parameters. Whether a DAA system is highly automated with human out of the loop or requires a more active human participant was not a consideration in defining sNMAC. The characteristics of a DAA system, such as the relationship between response times and alerting behavior, will influence whether a DAA system can meet a performance requirement, as defined by a set of metrics that include sNMAC frequency.

Formation Flight
Manned aircraft operating close to each other when in formation flight are not considered a NMAC. We make the same assumption for smaller UAS, and a DAA system evaluated using sNMAC-based metrics should not be penalized when operating in formation. Unlike NMAC, there is no quantitative separation criteria that defines formation flight. However, we assume that the general policy and guidance for manned formation flight will also apply to UASs in formation (14 CFR 91.111.b). Aircraft, either manned or unmanned, should not be considered in formation if each aircraft exhibits distinctive behaviors.
Quantitatively defining formation flight was out of scope and there was limited priori art to adopt. In simulation, Bower et al. [54] determined that an optimal formation for fuel savings was associated with aircraft separation based on 10% of the wingspan. Chao et al. [55] operated UASs with a desired separation of about 40 feet. While future work on defining formation flight is required, we assumed Bower et al. and Chao et al. maybe considered in formation .

Simulations
In Monte Carlo simulations with a one-second update rate, aircraft might transition through a sNMAC state between sequential updates. For niche cases like this, calculating if a sNMAC occurred via interpolation to shorter update rates is acceptable. While this is infrequent for NMAC, the smaller dimensions of sNMAC may make this simulation behavior more likely.

D. Sensitivity Analysis: Evaluation and Downselect
MIT Lincoln Laboratory (MIT LL) and the John Hopkins University Applied Physics Laboratory (JHU APL) collaborated to estimate the unmitigated likelihood of a MAC given a sNMAC potential candidate. A common experimental method was established. It was motivated by the unmitigated likelihood of a MAC depending on aircraft size and that an unmitigated NMAC was larger and more frequent than an unmitigated MAC. To enable a robust sensitivity analysis, each organization applied slightly different assumptions and distributions about aircraft size and separation between aircraft. The experimental sensitivity analysis consisted of these steps: 1) Select independent minimum and maximum limits for effective aircraft width and height 2) Select independent distribution types for effective width and height 3) Generate a set of horizontal and vertical miss distances 4) Sampling from the width and height distributions for each aircraft (#2) assess the unmitigated likelihood of MAC given horizontal miss distance and vertical miss distance (#3) 5) Visualize unmitigated P(MAC | sNMAC) given #4 6) Downselect sNMAC candidates that support unmitigated P(MAC | sNMAC) ≤ 0. 10

7)
Discuss potential sNMAC candidates at a forum held in February 2020. Participants included SMEs from MIT LL, JHU APL, and MITRE. No representatives from the FAA were in attendance. For simplicity, a MAC was assumed if the sum of the wingspans was greater than the minimum horizontal separation at closest point of approach and if the sum of the heights was greater than the vertical separation at closest point of approach Shared experimental assumptions included that aircraft are represented as volumes, each occupying a bounded, closed region of space and that aircraft position is a differentiable function of time. These shared assumptions aligned with Bellantoni [56] when they calculated aircraft collision probabilities. The volume assumption did not align with Kochenderfer et al. [5], who used wireframe models of aircraft to estimate P(MAC | NMAC). Both organizations also assumed the same minimum and maximum aircraft sizes, which is further discussed in the proceeding Section V.
The analysis was varied when selecting the distributions and distinct potential sNMAC candidates. One organization assumed a uniform distribution of horizontal and vertical miss distances; while the other simulated uniformly weighted encounters (see Section VII) and generated miss distance distributions based on the simulation results. Next, one organization evaluated unmitigated P(MAC) as a continuous function given horizontal and vertical miss distance and visualized this function as risk contours. Then potential sNMAC candidates were selected based on SME assessments of the unmitigated risk contours; this was functionally similar to Weinert et al. [2] who recommended well clear criteria for unmitigated small UAS encountering manned aircraft. The other organization estimated unmitigated P(MAC) from a SME selected finite set of potential sNMAC candidates.
The selection of potential sNMAC candidates was not sensitive to the experimental variations. Both organizations proposed similar sNMAC criteria given the shared aircraft assumptions in Section V. Given a choice between minimizing the vertical or horizontal dimensions, the forum had a preference to minimize the vertical. We assumed, although we did not validate, this would help facilitate vertical stacking of aircraft, promote aircraft remaining in trail, enable easier extension of hemispheric rules to UAS, and was overall be less disruptive to aircraft operations.

V. Smaller Unmanned Aircraft Assumptions
We defined smaller UAS based on aircraft size, not MGTOW, and analyzed the distributions of wingspan and height of smaller UAS. This is important due to the influence of aircraft size on NMAC and the recommended criteria for UAS-only encounters. As described by Kochenderfer et al. [5], the probability of a MAC is dependent on the physical size of the aircraft; during TCAS development, the NMAC was quantitatively defined based on the size of manned aircraft. Thus, defining which UAS are appropriate for sNMAC-based metrics was important.
Previously, Lester and Weinert [1] recommended that specific quantitative means for a UAS to remain well clear of manned aircraft should be applicable if the sum of wingspans for an encounter between a UAS and manned general aircraft were 100 feet or less. This was supported by an analysis using the FAA aircraft characteristics database 9 , indicating that the majority of general aviation aircraft had wingspans of 75 feet or less [2]. Thus, an UAS wingspan limit of 25 feet achieves the 100 feet sum of wingspan assumption for encounters between manned and unmanned aircraft. This wingspan scope has been adopted into the ASTM F38 WK62668 draft standard.
To align with the previous rationale [1] [2] [5], we defined "smaller UAS" as an UAS with a wingspan of 25 feet or less and without a MGTOW limit. This differs from "small UAS" which are defined by the FAA as UAS weighing less than 55 pounds MGTOW and aligning with 14 CFR 107.3. Since a likelihood of a MAC is significantly more dependent upon size than weight, quantitatively determining sNMAC applicability based on wingspan was appropriate. The material properties of the aircraft or specific engineering components were not considered.

A. Data Source
To characterize smaller UAS we analyzed the advertised specifications of UASs as documented by the Association for Unmanned Vehicle Systems International (AUVSI) unmanned systems and robotics database 10 . This database was used to support the preceding well clear research [1,2]. Table 1 reports that 1725 of 2438 (70%) active UASs have a wingspan less than or equal to 25 feet. As of November 2019, all active small UAS, as defined by MGTOW, had a wingspan less than 25 feet. By defining scope based on wingspan instead of weight, 489 more active UAS were included. For reference, there are only a few manned aircraft models with wingspans less than 25 feet.  Figure 1 illustrates the distribution of aircraft size organized by MGTOW , and Figures 2-3 illustrate the independent distributions of advertised wingspan and height. Notably there are many UAS with a MGTOW exceeding 55 pounds that have similar dimensions as UAS weighing less than 55 pounds. Independent of MGTOW, a vast majority of advertised UAS had a wingspan of 15 feet or less and a height of 6 feet or less. There was also a slight trend of an increased height as wingspan increased. Since sNMAC should not vary based on aircraft type, the size distributions were not organized into fixed-wing or rotorcraft types.

B. Distributions and Trends
These distributions provided explicit information about available aircraft and implicit assessments of the UAS market. The database cannot inform the relationship between total flight hours and aircraft size. Both the wingspan and height distributions skewed towards smaller dimensions. This suggested that market competition and subsequent market opportunity was greater for smaller aircraft. The assumed demand for smaller UAS was also supported by a previous characterization of commercial UAS operations prior to 14 CFR 107 [2] and more recently by UAS market research based on FAA aircraft registrations 11 . Specifically, there was strong evidence that an overwhelming majority of purchased and registered UAS with a maximum MGTOW of 55 pounds were manufactured by DJI. As of January 2020, all DJI UAS spanned less than 10 feet.
However, as the UAS market evolves and regulations change due to safety risk mitigations, larger UAS will likely be leveraged for operations that require relatively longer ranges or greater payload capacity. Example operations include the L3 HQ-90 12 , with a wingspan of about 16 feet and a MGTOW of 117 pounds, conducting a low altitude inspection of electric power lines 13 . The distributions of UAS size may also change and shift towards larger aircraft in response to urban air mobility operations, such as unmanned air cargo operations [57].

C. Minimums and Maximums
Based on the size distributions of smaller UAS, a set of sNMAC minimum dimensions were established. The horizontal and vertical dimensions were independently assessed and summarized by Table 2. Minimums needed to account for the tails of the wingspan and height distributions, as the sNMAC volume should support all modeling and simulation efforts in scope. These minimums were also mathematically symmetric, where both aircraft in the encounter would perceive the same loss of separation simultaneously. The symmetry property was also considered when evaluating UAS separation criteria [58]. Additionally, the NMAC dimensions of 500 and 100 feet were used as potential sNMAC maximums.
A horizontal minimum of 50 feet was based on the assumption that sum of smaller UAS wingspans should not exceed the sNMAC horizontal dimensions, given a maximum wingspan of 25 feet. 50 feet was simply the sum of wingspans for two such UASs. Coincidentally, 50 feet is 10% of the 500 feet horizontal dimension of a NMAC; as motivated in Section IV, candidates were not based on scaling the NMAC volume.
A vertical minimum of 15 feet was based on rounding effective height when banking or sum of most of heights for the larger UAS being considered. Although UASs with heights of 10-12 feet are currently rare, an aircraft with a 25 feet wingspan when banking can feasibly have an effective height greater than when flying straight and level. We assumed aircraft with extreme bank angles were rare and omitted them from consideration. Since aircraft size was not subject to the rounding consideration in Section IV.A, the maximum aircraft height was 12 feet.

VI. Potential sNMAC Candidates
As overviewed in Section IV.C, a distribution of horizontal and vertical miss distances were each sampled to define a set of close encounters to estimate the likelihood of an unmitigated MAC. Using a risk-adverse approach, an unmitigated MAC was only considered when both HMD is less than the average of wingspans and VMD is less than the average of heights. The unmitigated MAC estimation was significantly influenced by the chosen distributions of aircraft width and height.
To assess the sensitivity of the unmitigated MAC conditional probability, three distributions were evaluated for each aircraft width: uniform, left skewed, and right skewed. Illustrated by Figure 4, a uniform distribution represented an airspace with equal number of UAS with a given width. A left skewed distribution represents a higher concentration of smaller wingspans in the airspace, while a right skewed distribution represented the opposite concentration. The height distribution was uniform and did not vary. This was because the range of aircraft tail heights was smaller and no assumptions were made regarding distributions of aircraft bank angles. Contours of unmitigated P(MAC | sNMAC HMD & VMD) were calculated for each distribution to evaluate sensitivities and illustrated by Figures 6-8. Unmitigated P(MAC | sNMAC) were dependent on the assumed distributions. This dependence and subsequent variability suggested that using an overly conservative wingspan distribution may substantially increase the size of the sNMAC volume. Conversely in Figure Figure , a left skewed distribution pulls the unmitigated P(MAC | sNMAC) contours to smaller sNMAC dimensions. Most importantly in Figure 6, the likelihood of an unmitigated MAC was about 10%, given the sNMAC minimums of 50 and 15 feet. Historic assumptions about unmitigated P(MAC|NMAC) in manned aviation have used 10%, although Kochenderfer et al. [5], showed that actual values were often much smaller than the conservative 10% assumption .  The first potential sNMAC candidate was defined by the assumed minimums of a simultaneously loss of separation of 50 feet horizontally and 15 feet vertically. This candidate achieves the desired P(MAC) threshold of 10% or less. This MAC likelihood aligns with historic assumptions about unmitigated P(MAC|NMAC). Furthermore, it was assumed a left skewed distribution of wingspan is the closest representative of the current airspace. A substantial change in the airspace resulting in a uniform width distribution would not result in an unacceptable in crease in the MAC threat. Thus, the smallest potential sNMAC candidate of 50 and 15 feet could be robust to changes in the airspace. The alternative candidates meet the P(MAC) threshold for the right skew width distribution by either increasing the horizontal or vertical dimensions of the candidate #1. Potential sNMAC criteria assumes that all applicability requirements in Table 5 are satisfied by the UAS. As discussed in Section IV.A, there are no recommended restrictions based on altitude or airspeed. When used in simulation, sNMAC should be estimated based on the aircraft's center of mass in simulation. These requirements include the entire aircraft, so the effective height of a UAS transporting a payload attached to a tether would be the distance from the top of the airframe to the bottom of the payload. Since sNMAC is proposed to be a functional analog to NMAC, any of the potential sNMAC candidates should also be compatible with any regulatory or standards process that leveraged NMAC. The community adoption of final sNMAC criteria could lead to risk-based minimum operating performance standards (MOPS) for DAA systems [59,60] 14 . Upon selection of final singular sNMAC metric, it can be a "drop-in" replacement for NMAC, enabling the generation of a set of metrics used to train and evaluate ACAS sXu [53].

VII. Demonstrated Use in Simulation
As one purpose of sNMAC is to support modeling and simulation efforts, we developed a set of representative encounters between UASs to demonstrate that sNMAC statistics can be generated. Encounters were primarily simulated using the JHU APL RAVENS architecture, but also with the MIT LL CSIM and MIT LL CASSATT simulation environments. It was not tested using the MIT LL DEGAS environment. This demonstration showed that any of the potential sNMAC candidates can be integrated in simulations, leveraged for algorithm development, and provide results applicable to a safety assessment.
Due to the lack of operational flight logs to train a generative model, each trajectory was representative of expected commercial UAS operations and generated based on open source geospatial data [61,62]. Trajectory waypoints followed the terrain and tried to maintain a constant altitude with a tolerance of 25 feet. The initial separation between aircraft and encounter parameters were informed by an analytical assessment of all potential relative geometries between long linear infrastructure features for sixteen USA locations [63]. Table 4 documents the encounter configuration and aircraft parameters where airspeed was sampled from a uniform distribution. For each location, 10,000 latitude and longitude coordinates were sampled where pairs of different representative trajectories such that each trajectory was within 0.17 nautical miles of a coordinate. Trajectories were interpolated to have a fixed spacing of 50 feet between waypoints, and the WGS84 ellipsoid was used. For each pair, the distances between all combinations of waypoints were calculated. Ten waypoint combinations were selected s o that horizontal separation was 0.6 nautical miles or less 30 seconds after the start of the encounter. The 0.6 nautical mile threshold was based on a previous geometric assessment of long linear infrastructure [63]. For each combination, airspeed and altitudes were sampled for both aircraft and the trajectories were generated. Each combination had four potential trajectories where each aircraft could move forward or backwards, if sufficient track data existed. The tracks were not extrapolated with straight line segments to meet the time requirements. Table 5 reports the total encounters per state with about 300,000 encounters in total. Across all encounters, 76.8% had an unmitigated NMAC incident; 9.1% had an unmitigated loss of separation of at least 50 feet horizontally and 15 feet vertically (potential candidate 1 sNMAC). The quantity of encounters was comparable to previous assessments [2,64]. Since potential sNMAC candidate #1 was the smallest, these results demonstrated that any of the potential sNMAC candidates are implementable and that a sufficient quantity of encounters can be generated to evaluate the likelihood of an unmitigated sNMAC.
The encounter statistics across locations also indicated that encounter sets based on one or two locations may not be representative of the NAS and would be insufficient for DAA evaluations and safety case development. Because 10,000 coordinates were sampled for each location in an unweighted manner, the size of the state led to sampling bias. For example, Rhode Island was likely oversampled whereas California was under sampled. A complete assessment of a reference collision avoidance system would require additional encounters with a wider range of representative UAS behavior. This sampling issue was subsequently resolved after these simulations were completed [65]. The encounters were used to evaluate ACAS sXu version 2, a reference DAA system for smaller UAS. ACAS sXu version 2 reduced the likelihood of a sNMAC by at least 95% across all encounters when evaluated using JHU APL RAVENS. These rates were partly verified using MIT LL CSIM. While no performance requirements are set for a low altitude UAS only DAA system, these results indicate that potential requirements aligned with manned collision avoidance systems would be achievable. For more details on this, please refer to FAA TCAS Program Office documentation [9].
This also demonstrates that ACAS sXu can be designed to mitigate the likelihood of a loss of separation of at least 50 feet horizontally and 15 feet vertical. The ACAS sXu system is still being designed and standardized in coordination with RTCA, the aviation safety community should select a sNMAC criteria of at least 50 feet horizontally and 15 feet vertical prior to any ACAS sXu related final review and comment within RTCA. One consideration for the community could be the threshold where DAA algorithms become less effective due in further mitigating risk due to surveillance errors and other uncertainties.

VIII. Future Work and Conclusion
Motivated by the need to characterize the threat of collisions between UASs, we proposed qualitative and quantitative assumptions about sNMAC and investigated three of the smallest potential sNMAC criteria given these assumptions. We also demonstrated that the sNMAC candidates are appropriate for small UAS simulation studies.
In particular, sNMAC was designed to be a common, easily understood metric across the aviation safety community, similar to how NMAC has been used for encounters with manned aircraft. These contributions can lead to means to evaluate DAA system performance, support standards developing organizations, and develop aircraft encounter models. Additional mitigated risk analysis is required to determine if the sNMA candidates are suitable to evaluate safety and operational efficiency. A key assumption was the likelihood of a MAC given a sNMAC should be 10%; this assumption should be reassessed by future work. The presented research also focused on solely collision avoidance, and no explicit recommendations were made for considering well clear criteria for encounters between only UASs. Future research should also include assessments of well clear as well as sNMAC for encounters between only UASs for a holistic approach. The potential dependence and relationship between sNMAC and any well clear criteria should be explicitly considered and empirically assessed.
Specifically, these future efforts can address the needs of the ASTM Committee F38 for UAS or RTCA SC-147 for TCAS. We are expecting and welcome this paper's contributions to be iterated upon by stakeholders across the aviation community. Feedback on how the sNMAC assumptions and proposed criteria support real world current and future operations are of particular interest.
Further refinement and validation of the results could be achieved using a 3D wireframe model [5] with 6DOF dynamics to calculate both unmitigated and mitigated MAC conditional probabilities from various components of each aircraft. For example, such models could show the prevalence of one aircraft banking and its wing impacting the tail of another aircraft. Future work could also include the observability of sNMAC given expected smaller UAS altimetry and surveillance performance for specific equipage to validate usability. This surveillance-based future work was motivated by Semke et al. [66], when analyzing ADS-B performance, who identified deviations of 9-500 feet between barometric and geometric altitudes reported by the same aircraft equipped with ADS-B. Observability is an important operational consideration and for potential incident reporting.
Lastly, to promote a transparent and accessible technology transfer, much of the simulation and encounter sets described in this paper will be publicly released. They can be found at https://github.com/mit-ll or https://github.com/Airspace-Encounter-Models. Specifically, the MATLAB software used to generate Figures 4-8 has been released as "sNMAC-Initial [67]" under an open-source software license.