Human factors issues in the collocation of URET, TMA, and CPDLC

The Federal Aviation Administration (FAA) Free Flight Program individually deployed the user request evaluation tool (URET), traffic management advisor (TMA), and controller-pilot data link communications (CPDLC) to a limited number of air route traffic control centers (ARTCCs). Before deployment expands nationwide, it was important to identify any potential human factors issues that may arise due to the collocation of these tools at the controller's workstation. In this paper, we present the results of a high fidelity human-in-the-loop simulation we conducted to evaluate the impact of URET, TMA, and CPDLC collocation on air traffic controllers. We examined collocation issues with a "stovepipe" independent configuration where none of the tools were integrated or directly communicated with each other. Twelve certified professional controllers participated in the simulation working in two-person teams consisting of a radar (R-side) and data (D-side) controller. The most important collocation issue identified was that controllers had difficulty accessing important information on the D-side display when URET and CPDLC were both operational (i.e., display clutter). Although neither tool alone caused display clutter, both tools in combination made it difficult for D-side controllers to find the information they needed quickly. This was especially true for accessing CPDLC windows, which became covered when controllers used URET. Good human factors design principles prescribe that users must have immediate access to important information and that critical information should never be covered. A "stovepipe" independent deployment of these tools will result in impaired access to timely information. The results of this study indicated that better efforts should be made to integrate the information from URET, TMA, and CPDLC on the D-side monitor prior to deployment of all three tools at the controller's workstation.


Introduction
The FAA established the Free Flight Program in collaboration with the aviation community to increase capacity (airport and airspace) and improve efficiency (flight times and fuel consumption), while maintaining the current high level of safety. An important goal of the Free Flight Program was the delivery of new air traffic control (ATC) technologies focused on early benefits to users of the National Airspace System (NAS). These capabilities included URET, TMA, and CPDLC as en route controller tools. Under the Free Flight Program, these tools were individually deployed to a limited number of ARTCCs nationwide.
The MITRE Corporation developed URET as a conflict probe tool that automatically predicts and provides early identification of potential aircrafi-toaircraft and aircraft-to-special use airspace conflicts. URET has a trial planning feature that allows controllers to determine whether proposed flight path changes will conflict with traffic or restricted airspace. Controllers use trial planning to quickly evaluate possible route changes and to assign conflict free routing. URET is deployed on a flat-panel monitor and is operated by the D-side member of controller teams. URET consists of many different windows for displaying information such as aircraft flight data, graphical display of conflicts, and trial plans.
The National Aeronautical and Space Administration (NASA) developed TMA as an arrival sequencing tool that provides controllers with information for increasing the efficiency of traffic flow into airports. TMA uses time-based metering to sequence the flow of amval aircraft and replaces miles-in-trail as a method for aircraft spacing. Computer automation calculates the delay times for amving aircraft based upon aircraft type, flight plan, weather, and winds aloft and displays the times on a meter list. The TMA meter list is displayed on the R-side controller's Display System Replacement (DSR) monitor.
CPDLC is a tool that allows controllers and pilots to communicate with text-based electronic messages. CPDLC reduces the number of voice communications, which decreases radio congestion and reduces delays. On the flight deck, CPDLC messages are presented on a small monitor and pilots respond using buttons on the sides of the display. On the ground D-side, CPDLC Build 1A consists of three different windows for displaying uplinked messages and their status, a menu of text messages, and a history list. This version also has datablock fly-out windows and symbology for CPDLC messages. tools independently in a stand alone "stovepipe" design without a plan for integrating their system with other support systems. Therefore, we need to investigate how controllers will interact with the three tools before they are deployed together at the controller's workstation. When evaluating tools that will change or add to the number of systems used by controllers, it is important to identify any potential human factors issues that may arise from the introduction of these new tools. Identifying problems and comecting them before they can negatively impact performance in the field is critical in ATC where safety is potentially at stake. Therefore, the FAA Free Flight Program Office and the Human Factors Research and Engineering Division sponsored this study to examine the impact of collocating URET, TMA, and CPDLC at the controller's workstation.

Previous Research
A few studies have examined issues related to the collocation of URET, TMA, and CPDLC. Desenti, Gross, and Toma [ 11 examined the use of URET and TMA. The authors questioned whether there was an emerging concept of use for URET and TMA in which the trial planning capability of URET may be compatible to meet the delay times of TMA. A potential human factors issue identified was that URET and TMA used different algorithms to compute an aircraft's future location.
Kems [2] of the MITRE Center for Advanced Aviation System Development examined the human factors issues related to collocating URET and CPDLC. She concluded that D-side controller workload would increase, and the design of the Computer Human Interface (CHI) on the D-side was critical to the successful deployment of URET and CPDLC. Kems also noted there would be changes in the roles and responsibilities of both the R-side and D-side controllers.
Della Rocco, Panjwani, Friedman-Berg, Kopardekar, and Hah [3] conducted a cognitive walkthrough with subject-matter experts (SMEs) to explore the collocation of URET, TMA, and CPDLC. The study identified a number of CHI inconsistencies across the tools and raised questions about D-side display clutter with a "stovepipe" deployment of the tools. The authors concurred with Kems [2] in her assessment that the roles and responsibilities of the R-side and D-side positions need clarification when all three tools are deployed together. Furfher, applying human factors principles to these issues will help controllers use the tools as intended and attain the expected benefits.
To date, there have been no real-time humanin-the-loop simulation studies that have objectively examined controller performance while employing URET, TMA, and CPDLC. This paper describes the first experiment in a series of high-fidelity human-in-the-loop simulations conducted by the present authors to examine the human factors issues of collocating these three tools (for more details see [4 and 51). In this first experiment, we examined R-sidem-side controller teams working a high altitude sector and using different combinations of the three tools at a single sector. The second experiment examined controller teams interacting with each other while working a high and a low altitude sector and using all of the tools. The third experiment examined controllers working a high altitude sector with single controller staffing and using all of the tools.

Study Objectives
The specific objectives of the present study are To assess controller workload, To assess whether controllers have access to infomation when needed, situational awareness (SA), and teamwork, and To identify any other important human factors issues.

Particzpanis
Specialists (ATCSs) from Level 11 and Level 12 ARTCCs nationwide participated in this study. We recruited six participants who were URET current and proficient from ARTCCs where URET is operational. We recruited six participants who were TMA current and proficient from ARTCCs where TMA is operational. We trained all twelve ATCSs in CPDLC after arriving at our research facility. Also, all participants received some cross-training in URET and TMA. Each controller team consisted of one TMA-qualified ATCS operating the R-side and one URET-qualified ATCS operating the D-side position.

Controllers completed a Background
Questionnaire to describe the general demographic characteristics of participants in the study. The mean age of participants was 40.52 (range 28-47) years old with a mean of 15.10 (range 5-21) years of FAA experience. All participants actively controlled traffic for the past 12 months.

Simulation Equipment and Setup
William J. Hughes Technical Center in the Research, Development, and Human Factors Laboratory. Each R-side workstation consisted of a high-resolution Sony 2K monitor, DSR keyboard, and trackball. We deployed TMA and CPDLC on the Sony 2K monitor. Each D-side workstation consisted of a 21" flat-panel monitor with keyboard and mouse. We deployed URET and CPDLC on the flat-panel monitor. The voice communications system consisted of individual relay switchboxes, controller headsets with microphones, and push-totalk handsets or foot pedals. Flight strip marking was optional in our simulation and only one of the controller teams requested and marked flight progress strips.
An SME observed each controller team and provided ratings and comments on controller interaction with the tools. A team of simdation pilots communicated with contrdlers using proper ATC phraseology and maneuvered aircraft using simple keyboard commands.
We conducted the simulation at the FAA

Airspace and Traflc Scenarios
We selected a generic high altitude en route sector as the airspace for this simulation. SMEs designed the traffic scenarios with moderate traffic levels that required traffic flow restrictions, but were not so busy as to overwhelm controllers who were not experienced with all of the tools. We prepared one basic test scenario and designed seven additional test scenarios based on the basic scenario. All scenarios had the same number of arrivals, deparhres, and overflights; however, we changed the aircraft entry times and assigned different callsigns in each of the scenarios. This ensured that the test scenarios were similar in traffic, but not recognizable as the same scenario. Each test scenario was 45 minutes in duration.
in scenarios were equipped. CPDLC services included Transfer of Communication (TOC), Initial Contact (IC), Altimeter Setting (AS), and Menu Text Messages (MT). In manual TOC mode, CPDLC will generate a held TOC message that must be released by either the R-side or D-side controller for the system to uplink a new frequency to aircraft. We allowed the ATCSs to decide for themselves which team member would release aircraft with held TOCs.
When CPDLC was in use, 40% of the aircraft

Experimental Design and Measures
The experiment represented a two-factor design with R-side and D-side controller positions as the first factor and eight different tool combinations as the second factor. The eight tool combinations are represented by U, T, C, UT, UC, TC, UTC, and No Tools as a baseline where U=URET, T=TMA, and C=CPDLC. Each team of controllers completed eight test scenarios with each scenario representing a different tool combination.
We collected a large set of subjective measures that included controller workload and SA, SME ratings and observations, and participant questionnaires. We also collected a large set of objective measures for ATC simulation research that included safety, capacity, efficiency, and communications indicators [6]. For this paper, we selected a few measures that were directly influenced by the tools. Table 1 presents these simulation measures.

Procedure
The study consisted of three testing sessions, each lasting two weeks with different groups of four ATCSs. In the first week, we briefed participants on the study objectives, airspace operations, and support tools. Controllers completed 18 hours of training scenarios to become proficient with the generic airspace and familiar with all three support tools. In the second week, we began data collection and controllers completed a series of test scenarios. At the end of each day, we had a group meeting with controllers to discuss the issues raised during the simulation. scenarios. We counterbalanced the presentation order of the scenarios and tool combinations to experimentally control for practice effects. At the start of each test day, controllers performed a preliminary warm-up scenario followed by four test scenarios. Before each scenario, the researchers informed the participants which tool or tools would be operational. Although there were two teams of controllers participating simultaneously, each team independently controlled different traffic scenarios in the high altitude sector. The sectors adjacent to the high altitude sector were automated by "ghost" controller functionality.

Results
We used univariate inferential statistics to analyze the data in this study. For each simulation measure, we performed a two-way analysis of variance (ANOVA) with Position (R-side, D-side) as a between-subjects factor and Tools (None, U, T, C, UT, UC, TC, and UTC) as a repeated measures factor. In each ANOVA, the standard significance level w a s p c .05. When there were significant effects, we performed an analysis ofthe simple effects and Tukey HSD post hoc comparisons to determine which means were different using a group comparisons significance level o f p < .05. Table 2 shows controller ratings for how often CPDLC caused clutter on the R-side and D-side displays for each of the four tool combinations with CPDLC operational. R-side and D-side ratings were different depending upon the tools in use,

Access to Tool Information
[Position x Tools: F(3,30) = 4 . 3 6 ,~ = ,0121. R-side controller ratings were rather low indicating that they did not think CPDLC caused clutter very often. D-side controller ratings ranged from low to moderate depending upon the tools in use, [D-side Tools: F(3,15) = 4 . 0 3 ,~ = ,0271. When CPDLC was displayed alone and when CPDLC and TMA were displayed together, ratings of display clutter were very low. Ratings were much higher when CPDLC and URET were both displayed and when all three tools were displayed together (confirmed by Tukey HSD comparisons).  Table 3 shows controller ratings for how often URET caused clutter on the D-side display for each of the four tool combinations with URET operational. In this case, D-side controller ratings were rather low indicating that they did not think URET caused clutter on the D-side display whether used alone or in combination with TMA and CPDLC.

D-side
In our group discussions, controllers frequently commented that when URET and CPDLC were collocated on the D-side, there was display clutter.  Some controllers commented that they wanted TMA delay times available on the D-side display. In the simulation, R-side controllers positioned the TMA list in a location near the edge of the DSR monitor where D-side controllers could see the information. The participants commented that the R-side sliould be able to position the TMA list wherever is most convenient for h i d e r and that D-side controllers should have access to this information on the D-side display. Controllers were concerned that simply showing the TMA list on the D-side monitor would add to the display clutter problem, but thought there should be a smart way to show TMA information on the D-side position. Controllers suggested that it would be better if TMA delay times were available on the URET Aircraft List where they would not increase display clutter.

5.B.5-5
Controller Workload and SA demand workload ratings provided by the R-side and D-side positions for each of the eight tool combinations. For R-side controllers, mental demand ratings were only moderate and there were no differences between the tool combinations. For D-side controllers, mental demand ratings ranged  R-side controller ratings ranged from 6.67 to 8.83 and D-side controller ratings ranged from 7.50 to 9.00 in the eight tool combinations. Table 4 shows the mean number of CPDLC TOC messages sent by R-side and D-side controllers for each of the four tool combinations with CPDLC operational. The R-side sent from 90% to 95% of the CPDLC TOC messages in each of the tool combinations [Position: F(1,IO) = 194.96,~ < .001]. Therefore, the controller teams controllers had to visually monitor the CPDLC Message Out window to know when their team member sent a message or verbally coordinate with each other when messages were sent. This may become a multi-tool issue when one controller sends a CPDLC TOC message early and, based on URET or TMA information, the other team member decides to take (or recommends) action on the aircraft.

Discussion
The most important human factors issue identified in this study was that controllers had difficulty accessing important information on the D-side display when URET and CPDLC were both operational (i.e., display clutter). Controller ratings indicated that the CPDLC windows on the D-side monitor were the source of this display clutter problem. However, all the D-side controllers were accustomed to using URET as a stand alone tool at their own facilities and none had worked with CPDLC prior to this study. Therefore, it is not surprising that these controllers would attribute the D-side display clutter to CPDLC and not URET. Controller ratings indicated that neither URET alone nor CPDLC alone caused D-side display clutter. Finally, R-side controller ratings indicated that CPDLC caused very little display clutter on the DSR display.
This D-side display clutter problem resulted from the "stovepipe" independent deployment of both URET and CPDLC and having to manage the multiple windows associated with these tools. It is important to note that we developed the D-side CPDLC CHI for use in this study to be consistent with the "stovepipe" independent deployment of these tools. We added simple CHI features to help controllers manage the multiple windows associated with each tool. SMEs from the CPDLC users group helped us develop the D-side CHI. At our request, the FAA's Air Traffic Design Evolution Team (ATDET) approved the interface for use in this study. However, this specific D-side CHI was not intended to be the interface that will be deployed to ARTCCs in the future.
Another collocation issue identified in this study was that D-side controllers had to access TMA delay time information on the R-side display. In the simulation, R-side controllers positioned the TMA list in a location near the edge of the DSR monitor where D-side controllers could see the information. Both controllers spent a great deal of time viewing the TMA list. Controllers thought it was important to have TMA information available on the D-side display where it could be more easily accessed by D-side controllers. However, controllers were concemed that simply showing the TMA list on the D-side monitor might add to the display clutter.
Good human factors design principles prescribe that users must have immediate access to important information and that critical information should never be covered. A "stovepipe" independent deployment of these tools will result in impaired access to timely information. The results of this study indicated that better human factors efforts should be made towards integrating the information from URET, TMA, and CPDLC. Even if these systems cannot be entirely integrated, we should explore integrating the information displays on the D-side monitor prior to deployment.
Controller NASA-TLX workload ratings indicated that R-side mental demand was only moderate and did not change with tool use. D-side mental demand ratings tended to increase when two and three tools were operational. However, D-side workload ratings were only low to moderate and never reached an excessively high level. We designed the scenarios with moderate traffic levels so that controllers would not be overwhelmed with traffic and stop using the tools. We thought moderate traffic levels would allow controllers to use the tools more often and better identify any collocation issues. Therefore, the workload results of this study may have been different with higher trafic levels and greater workload demands on controllers.
the baseline condition without any tools. Workload for this baseline condition may have been lower than actual field conditions where flight progress strips are actively used because five of the six controller teams did not want flight strips posted at their workstation. All of our D-side controllers came from URET facilities that do not require flight strips. The baseline condition was intended to be an experimental comparison for the conditions with D-side controller workload was rather low in

5.B.5-7
tools. Without flight strips, controllers used flight plan readouts to obtain aircraft routing information.
Controllers rated their SA as very high and did not vary with different tool combinations. Although sometimes self-ratings can potentially misrepresent true SA, there was no indication that this actually occurred during the simulation. With very few exceptions, controllers maintained safety throughout the scenarios. For the present study, we used self-ratings of SA as a technique to elicit controller concems and identify collocation issues for group discussion.
indicated that R-side controllers sent most of the TOCs. There were no mandatory procedures for using CPDLC by the R-side and D-side in our simulation. We allowed each controller team to practice together and decide for themselves who would send the CPDLC TOCs to aircraft. Although the results indicated that D-side controllers did not use the TOC service very often, controllers still expressed concems about not knowing what their team member was doing with CPDLC. Unlike voice communications, there were no audible cues with CPDLC to help controllers maintain SA of their team member's actions. Controllers had to visually monitor the CPDLC Message Out window to know when their team member sent a TOC message. If the CPDLC windows were covered by URET, the D-side controller could miss a sent message. In addition, CPDLC messages were visible in the Message Out window for only a few seconds after a pilot "wilco" was received. Therefore, controllers had to be very vigilant. An even greater concem for safety exists if CPDLC is used to issue control instructions (e.g., altitude, heading or airspeed future services). It is possible that D-side controllers could recommend actions that create aircraft conflicts when they are not aware of the CPDLC messages sent. More research needs to be conducted examining the roles and responsibilities for CPDLC usage, especially when other tools are being used. study was to identify human factors issues when URET, TMA, and CPDLC were collocated together at the same sector. In a high-fidelity human-in-theloop simulation, the main issue was D-side display clutter when URET and CPDLC were being used Finally, the number of CPDLC TOC messages In conclusion, the purpose of this simulation together. Another issue was the need for TMA information on the D-side display without adding to the display clutter. From a human factors perspective, URET, TMA, and CPDLC should be integrated on the D-side display. This solution avoids the problem of multiple windows or displays that were a consequence of the "stovepipe" implementation of these tools in our simulation. With URET, TMA, and CPDLC information integrated, controllers should have easier access to information when needed without having to monitor and manage multiple displays.
Controllers also provided some non-integration ideas including a larger D-side display and an improved D-side CHI that would make it easier to manage multiple information displays. We recommend that additional research be conducted to investigate URET, TMA, and CPDLC collocation issues and potential solutions. Future simulations should examine the best presentation of the information, specific procedures for R-side and D-side tool use, and higher traffic levels for scenarios.