Line pilot perspectives on complexity of terminal instrument flight procedures

Many new Performance Based Navigation (PBN) Instrument Flight Procedures (IFPs) are being developed as the United States transforms its airspace to improve safety and efficiency. Despite significant efforts to prepare for operational implementation of new IFPs, the process does not always go smoothly. The primary goal of this study was to understand what makes IFPs difficult from the perspective of line pilots. We spoke to 45 professional pilots in small groups. The pilots reviewed, briefed, and discussed six IFPs in an office setting. We extracted a comprehensive list of subjective complexity factors by observing pilot briefings and gathering pilot feedback. Then we organized the list into a framework that captures a variety of types of complexity. We define a subjective complexity factor as one that requires an extra mental or physical step by the pilot. IFP design parameters (e.g., the number of transitions and flight path constraints) are a main driver for subjective complexity for line pilots. Unusual IFP designs can result in novel chart depictions that are unfamiliar and more difficult to use. In turn, novel chart formats may have inconsistencies that increase subjective complexity. Participants also mentioned factors that are outside the control of IFP designers, such as weather, fatigue, and aircraft performance or equipment. We separate out these as operational complexity factors. The broad nature of the pilot interviews also provided insights into how pilots use charts today, in the context of the modern flight deck. A full report on the study is in preparation.


I. INTRODUCTION
Airport terminal airspace in the United States is undergoing a major transformation. Many Instrument Flight Procedures (IFPs) are being developed and implemented to streamline operations in and out of airports. An IFP is defined in [1] as "a charted flight path defined by a series of navigation fixes, altitudes, and courses provided with lateral and vertical protection from obstacles from the beginning of the path to a termination point." New IFPs enhance safety and efficiency through the use of Performance Based Navigation (PBN) [2]. PBN is based upon Area Navigation (RNAV) and Required Navigation Performance (RNP), which provide more accuracy and precision than conventional, ground-based, navigation aids. PBN IFPs include RNAV Standard Terminal Arrivals Routes (STARs), RNAV Standard Instrument Departures (SIDs) and Instrument Approach Procedures (IAPs) that use RNP. For data on the use of such procedures, see www.faa.gov/nextgen/pbn/dashboard.
There are many stakeholders in the development of new PBN IFPs, including: • IFP designers who develop the flight path instructions; • Regulators who ensure that the IFP design meets established criteria and specifications; • Operators who seek particular types of safety and efficiency from the new IFP design; • Cartographic designers who convert the IFP design into a visual representation, a chart, for use by the flight crew; • Air Traffic controllers who assign the IFPs to flight crews and confirm their progress along the assigned route; • Flight crews who fly the IFPs; • Aircraft-systems designers whose avionics and software are used by flight crews to fly the IFP.
All of these stakeholders share common goals. First, it is important to give pilots clear instructions on the route to fly while conveying any restrictions on speed and altitude. Second, new IFPs should allow flexibility for efficiency of the airspace and accommodate a diverse assortment of aircraft types. And finally, the IFP design should consider human performance, workload, and the potential for confusion.
Although new IFP designs are vetted carefully through design criteria [3,4] and software and flight simulations [1,5], operational implementation does not always go smoothly. This is not necessarily a direct result of PBN itself, but may have more to do with how PBN is being used. Shortcomings of new IFPs are often blamed on "human factors," but, there are many humans involved and each has his/her own perspective on the difficulties they face. For example, IFP designers may think of complexity in terms of parameters such as vertical and lateral flight paths, but pilots-particularly line pilots-may think differently about complexity.
Human factors considerations for PBN were first reported systematically by Barhydt and Adams in 2006 [6]. Their list of issues is comprehensive and has many interrelated topics that cut across different stakeholder groups. This technical report [6] and a conference paper by the same authors [7] concluded that there was a need for specific instrument procedure design guidelines that consider the effects of human performance.
In [7], researchers analyzed reports from the National Aeronautics and Space Administration (NASA) Aviation Safety Reporting System (ASRS) to understand pilot issues related to RNAV SIDs and STARs. They found that pilot issues could be traced back to Air Traffic procedures, airline operations, aircraft systems, instrument procedure design, or some combination of these factors. An updated look at ASRS events related to RNAV operations [8] came to a similar conclusion, that operational issues arise from a combination of factors related to Air Traffic, aircraft equipment, and instrument procedure design. In addition, [8] grouped instrument procedure design with chart design issues because ASRS reports do not provide sufficient information to distinguish between procedure and chart design.
Two others studies explored the complexity of PBN IFPs [9,10]. In [9], we attempted to quantify procedure complexity objectively. We found that the complexity of a PBN IFP is difficult to measure with only quantifiable parameters. The analysis yielded only general findings that were relatively unsurprising. IFP design parameters that were associated with an increased number of ASRS reports for STARs were (a) more path segments (i.e., waypoints) and (b) more altitude constraints. For SIDs, an increased number of ASRS reports was associated with IFPs that had more flight paths (i.e., transitions). The IAPs analyzed in [9] were selected by subject matter experts, not for known operational issues, but for perceived issues. The group of IAPs selected by experts had more flight paths, more path segments, and more radius-to-fix path segments than a baseline group.
We also realize that IFP design can negatively impact visual complexity of the chart because the IFP design affects how many flight paths and other elements (e.g., notes) need to be shown. In [10], we established empirically that pilots need more time to find specific information when the chart is visually complex, with many transitions on SIDs and feeders on IAPs. Although pilots may feel that they ignore irrelevant flight paths when searching for specific chart data our results show otherwise. The time to search for an item on the chart was affected by all the flight paths shown.
Having explored objective measures of IFP design complexity and visual complexity of charts, we now turn our focus to subjective complexity of IFPs. Because charts depict the IFP design, we needed to investigate both IFP and chart complexity, together, in context, and separately in order to understand subjective complexity fully.

II. METHOD
We spoke to 45 professional pilots in small groups. The pilots reviewed, briefed, and discussed two STARs, two SIDs, and two IAPs in an office setting. The pilots also described their aircraft equipment and flight deck displays, and explained how these systems affected their handling of different IFPs. We took detailed notes to record the session. No quantitative performance data were recorded.
The open ended discussion allowed pilots to comment more generally, allowing us to learn about how pilots use charts and review IFPs today, in the context of aircraft with Flight Management Systems (FMSs). We wanted to understand how pilots prepare to fly an IFP to examine whether there is room for improvement in this process, either in terms of pilot technique, or IFP/chart design.

A. Participants and Their Aircraft
All participants were certificated and current professional instrument-rated pilots. We recruited volunteers for the study through advertisements internal to the different operators. Pilots received a $200 gift certificate for their participation if allowed by their employer.
We met with groups of two or three line pilots for three hour sessions. 1 Each group was from the same operator and flew the same (or a highly similar) type aircraft. A total of 45 professional pilots participated; 23 were qualified for RNP Authorization Required (AR) operations, the rest were not. The pilots came from three major airlines, a regional airline, an air taxi operator, and three corporate operators.
Participants' flight experience ranged from 3,200 hours to over 22,000 hours. All of the participants used Jeppesen charts in their daily operations. Some used electronic chart viewers, but not all. Most flew domestic operations in the United States, but some had experience flying in Europe. We did not require Captains to be paired with First Officers.
The aircraft types that the participants flew came from different manufacturers. All of the aircraft flown by participants were equipped with FMSs. Participants selfreported high levels of experience with FMS equipment. All but two (one corporate and one regional pilot) reported 5 years FMS experience or more. Some of the aircraft the participants flew had auto-throttle, some did not. Some had advisory Vertical Navigation (VNAV) systems, while others had full VNAV, which can be coupled to the autopilot.

B. Session
We conducted the interviews in conference rooms. All sessions followed the same script. First, we introduced participants to the purpose of the study and its structure, then asked them to sign informed consent forms. Next, we explained the process for discussing each IFP. After looking at all the IFPs, we concluded with a discussion of IFP and chart complexity in general.
We took detailed notes of the conversation on laptop computers independently. We did not record audio or video of the interviews. Because they were not in a simulator, participants had to clearly explain in words how they would fly the IFP. We interrupted with questions if needed. No performance data were gathered. We did not record the time spent reviewing each chart or the accuracy of the briefing. We also did not record any interactions with the flight deck systems because those were not available.

C. Stimuli
The six IFPs tested in the study are listed in Table I, grouped in two sets. The order of the two sets was switched between pilot groups to balance the potential effects of fatigue. All but one of the IFPs required use of RNAV. We included a conventional arrival for comparison (the KORRY THREE arrival into La Guardia airport). Pilots who were qualified for RNP AR saw the Boise RNP AR approach, and pilots who were not qualified saw the Boise GPS approach. Two EDETH SID versions were used because the procedure changed during the testing period. There were only minor differences in the new version, but we wanted to use current charts only.
We provided full size paper copies of the familiar Jeppesen charts for the pilot reviews and briefings. We also had an iPad with Jeppesen charts if they preferred to use that. However, the focus of the study was on the IFP and chart design, not the media that the pilots used (electronic or paper).

D. Task and Fidelity
Participants reviewed each IFP in stages. First, they completed a silent individual review then did a group review (briefing) when they were ready. Participants took turns leading the briefing. After reviewing and briefing the IFP, pilots completed the last two steps, first answering general questions, then specific questions about the IFP. Pilots normally do the first two steps (silent review and group briefing) in the flight deck while preparing to fly the IFP. We gave them as much time as they wanted without interruption. We asked them to do what they would "normally do," though we recognize that, without their flight deck systems in front of them, participants had to mentally recreate their normal process.
Our goal was to strike a balance between the formality and informality of the review and briefing tasks. We did not want the pilots to feel they had to demonstrate ideal or perfect performance, but we also did not want them to do the tasks so casually that they missed features they would normally examine. Grouping participants from the same operator helped to make the situation more realistic without undue formality.
One artificial aspect of the study was that we did not give participants a clearance, which they would normally have. Instead, they decided what route to brief and could choose to review more than one route if desired. We wanted to observe what they chose to do.

E. Limitations and Scope
The interview method had limitations. In particular, we did not observe pilots flying the IFPs. We only talked about how the participants would fly them, so we do not have flight performance data. Because we were in a conference room, participants could not access an aircraft simulator or a mockup of flight deck displays, though, sometimes, we were able to open up a view of their flight deck in a paper diagram or an online picture for reference. Also, data from the interviews reflects a group opinion. We could not separate out individual participant responses. This, however, reflects reality because two crewmembers share their knowledge in the flight deck.
There were also limitations to the set of data we gathered. Our data are from a broad sample of pilots who flew a variety of aircraft types. However, the sample is not deep. We would need to collect additional data if there are questions that focus on a specific subset of our participants or the aircraft they flew. None of the participants flew light general aviation, singleengine operations. Our focus was on commercial operations and pilots who have significant experience with RNAV operations.
Finally, the scope of the study was limited. The data we gathered were focused on the IFP design and the chart. We did not examine other factors in understanding and flying the IFP (e.g., the role of the FMS, Air Traffic clearances, and pilot training). Also, we did not systematically compare how pilot briefings varied based on whether participants used an Electronic Flight Bag (EFB) or the paper chart. All of these topics could be considered in future studies.

III. ANALYSIS
Interview notes from both researchers were combined and summarized into a common format using a rubric. We constructed the rubric based on an initial subset of the data, then applied it to data from each of the participant groups. The result was a single summary data file for each participant group, organized by topic. We then collated data across groups for the different topic areas to capture response patterns and variations. Example topic areas in the rubric included: Briefing Techniques, In-flight Techniques, Aircraft Equipment, General Comments/Questions, General Procedure Comments, and Chart Format Comments. The summary file also had responses to two questions we asked at the conclusion of the study: What makes an instrument flight procedure difficult? and What makes a chart difficult to use?.

IV. RESULTS
We obtained two types of data. First, there were our observations about how pilots completed their individual reviews and briefings. Second, we had participant responses to our discussion questions. In many cases, their responses were far ranging and covered more subject matter than was directly mentioned in the question. For example, participants often described what they would do in a variety of circumstances (e.g., in poor weather, or with an unfamiliar crew member), or what they might do in other aircraft that they have flown.
Here, we first discuss how participants completed their individual reviews and group briefings (Section A), comparing the process we observed to recommendations from other sources [11,12]. In Section B, we present participant feedback about operational complexity factors, such as interventions by Air Traffic Control (ATC). In Section C, we present participant feedback on some factors they believed created difficulty in flying IFPs or using charts. Space limitations preclude a complete discussion of all findings. In Section D, we describe our subjective complexity framework for line pilots.
A full report (in preparation) covers each of these topics in more detail and has chart images of all the IFPs tested. The full report also discusses interactions between aircraft systems, IFP design, and operational complexity factors.

A. IFP Reviews, Briefings and Use of Chart Data
When given an IFP and chart without a clearance, the first thing our participants did was to create their own clearance. They reviewed only the flight path for that clearance. That clearly answered our first question as to whether pilots oriented themselves to the entire IFP, or just to one path. Only a few participants reviewed the IFP beyond the intended route of flight.
Lutat and Swah recommend a three-step process for preparing to fly an IFP in an aircraft with an FMS [11]. The first step is to " 'build,' load, or select the appropriate procedure from the FMS database with direct reference to the current chart." The next step is to "rigorously check the FMS programming for compatibility with the aircraft clearance." And finally, "The pilot flying (workload permitting) briefs the procedure from the FMS, while the pilot monitoring (ideally) checks the programming against the appropriate chart(s)." In addition, [12] points out that effective briefings need not take a long time. Just a minute or two can be sufficient if the pilot has carefully selected the most important points to cover.
Participants described a process that was similar to these recommendations. Some took more time than others to review the charts silently, but their briefings were generally completed within a minute or two. A few participants wrote notes or calculations on the chart during their review, or followed the flight path with a finger or pen. This was their opportunity to build a mental representation of the flight path.
Most participants said that one pilot typically reviews the IFP and chart to ensure that it is properly entered in the FMS. This pilot combines information about the flight path from several sources (e.g., the route in the FMS, the filed route, the cleared route, the chart, the airport terminal information system, and from their prior familiarity with the local area). As this pilot reconciles the different sources of route information, he/she becomes familiar with it and identifies special features that the other pilot may need to know. It is particularly important to verify that the route entered in the FMS and the charted IFP match because the chart is the legal authority, but the path in the FMS is the one the aircraft will fly.
Good briefings are generally short, precise, and thorough. Typically, the pilot who entered the route into the FMS tells the other pilot about the planned flight path. In our study, participants briefed speed and altitude constraints in detail, particularly if they planned to use their automated navigation systems. Briefings are conducted out loud and they are structured, but participants told us that they still vary based on factors such as familiarity with the IFP, crew familiarity with each other, and weather conditions. We were also interested to know what elements of the chart participants used for their reviews and briefings. Although the study was not designed to systematically address this topic, we gathered some data by observing pilots use the charts to brief the IFPs and by asking pilots what they would do in general. Table II provides a summary of results on the use of chart elements. Some elements were used consistently, others were used sometimes, and a few were used rarely. Notice that Table  II does not specify when pilots used a particular element. The element could be used during a pilot review, crew briefing, or in flight. These results may be useful when developing datadriven terminal charts, which could be customized for different aircraft or operators. Keep in mind, however, that our data are from professional pilots based in the United States, who fly RNAV regularly. These data may not reflect the needs of other types of pilots and operators.

B. Operational Complexity Factors
Participants mentioned several factors (which they said created difficulties) that could not be controlled by the IFP designers or chart designers. These factors are sources of "operational complexity." We grouped them into five categories: • ATC interventions. For example, route amendments (either late or not), unpublished restrictions, or vectors.
• Aircraft equipment or performance factors. These include lack of or unreliability of automated systems, and performance characteristics of the aircraft.
• Environment factors. Examples include traffic, weather (winds or instrument meteorological conditions), terrain, and prohibited airspace. • Operator factors. These include operator policies on when or how to use flight deck automated systems, reliance upon dispatchers, and the clarity of pilot-flying and pilot-monitoring roles in reviewing, briefing, and flying IFPs.
Air Traffic interventions can be difficult to handle for flight crews. An amended route issued with little time for the crew to reprogram the FMS, review and re-brief the new path creates high workload. Other types of real-time air traffic interventions increase communication workload, and can be confusing to crews. They can even negate the advantages of flying a published IFP. For example, one pilot group mentioned that they were set up for an RNAV (RNP) AR approach only to be given a visual approach at the last moment. While the visual approach was operationally efficient, they would have preferred to practice the RNP AR procedure in good weather.
Aircraft equipment or performance is an issue when systems are unreliable or have unusual operational characteristics. The reliability of their VNAV system is especially important. Our participants said they make choices about how and when to use coupled and advisory VNAV for each IFP. Some participants used VNAV only for descents, not climb. Some participants chose to use coupled VNAV in advisory mode because of their lack of confidence in the autopilot. Some participants mentioned that operating VNAV creates set up and monitoring work. It does not fully relieve the pilot.
Some environment factors are dynamic. Traffic, for example, can vary based on the time of day. Weather varies too, sometimes dramatically. Although IFP designs take into account historical wind patterns, if actual winds are beyond the norms, the IFP may be difficult to fly. Terrain and prohibited airspace are considered in the IFP design too. In theory, if the aircraft stays on the specified flight path, it is not necessary for the pilot to be aware of nearby terrain or prohibited airspace. However, our participants indicated they were especially aware of steeply rising terrain (e.g., near the EDETH departure) or specific prohibited airspace, such as the area around the White House that is close to the FRDMM arrival flight path.
Flight crew factors are affected by training and level of experience. There can also be individual differences based on personal styles. We found, for example, that some participants did more thorough individual reviews than others. Another crew factor is familiarity. Some participants adjusted their briefings if they were familiar with each other.
Operator factors varied between corporate, air taxi, and regional, or major airline. For example, operator policies may vary in ways that affect how the IFP is flown, even for the same general type of aircraft. In a given airspace, any of these types of operators, or a mix, could be flying the same IFP.

C. IFP and Chart Design Difficulties
At the end of the study, we asked participants what makes an instrument flight procedure difficult, and what makes a chart difficult to use. Participant responses indicated that they did not clearly distinguish between IFP complexity, chart complexity, and operational complexity (described in Section B). For example, participants mentioned IFP design-related issues such as notes, variability, and waypoint names in response to the chart difficulty question. Participants mentioned operational factors in response to the question about IFP difficulty. We detangled the responses that participants provided.
To understand IFP design and chart design issues better, we combined participant responses to the two final questions with data from other sections of the summary files for each pilot group. In the full report, we provide many specific examples of issues that participants either brought up, or that we observed with the sample charts in our study. Here we provide just a few examples and then present the line pilot subjective complexity framework.
After separating out operational complexity factors, we found that subjective complexity arose from many sources related to both the IFP and the chart. We define a subjective complexity factor as one that requires an extra mental or physical step by the pilot. These factors create an increased need for memory, awareness, attention, or even additional physical actions. If participants mentioned that they would be especially aware under certain circumstances, or especially careful, these were our cues that there was subjective complexity. Some examples include: • Ambiguity • IFP variability (and related chart variation); • Depiction of non-contiguous paths (e.g., using insets, multiple pages, or panels); • Multiple transitions; • Waypoint names; • Notes and their depiction; • Problem connections (e.g., an unexpected rapid change in the rate of descent between IFP segments); • Speed and altitude constraints.
Ambiguity of the chart or IFP design is a subtle factor because it is in the eye of the beholder. An IFP that is ambiguous to one group of pilots may not be ambiguous to another. The source of ambiguity could be the IFP design, or it could be a lack of training about how to interpret the IFP design, or a lack of understanding about how an aircraft system would work under particular conditions, or an unclear chart depiction. One example of ambiguity in our chart sample was in the Denver ILS procedure, which has RNAV transitions to an ILS. Some participants who had never flown in that airspace were confused by the RNAV transitions. They did not know what the altitudes along the transition segments were. Were they minimum enroute altitudes (as on a SID/STAR), or mandatory altitudes the aircraft had to reach before the next waypoint with a lower altitude? Some participants also were not sure whether they would get a separate clearance to join the localizer at Denver after they had flown the RNAV transitions.
Flight path constraints are an especially large source of complexity. From the pilot's point of view, every constraint has to be monitored carefully for compliance, and this can create workload both during the pilot review and briefings and in flight. Flight path constraints were discussed frequently for the FRDMM STAR in the study, which is an optimal profile descent arrival with many altitude and speed constraints of different types. Although some participants felt it was easier to have altitude and speed constraints at each waypoint, most felt the opposite, that having a less constrained flight path was easier to fly. Some of the aircraft that participants flew did not handle vertical flight path constraints well, creating additional flight crew workload to monitor and adjust the vertical path. Lateral flight path constraints tended not be an issue for these participants and their aircraft; the participants trusted their aircraft would manage the lateral path well.
Waypoint names also impacted participant briefings. The participants stumbled over unusual names or just read them incorrectly. Participants spelled out names such as ICUJY and IDOCY (on the Boise RNAV (RNP) AR IAP), which slowed down the briefing. SEPII (on the FRDMM STAR) was also a confusing name; some participants called it the "September 11" waypoint, while others called it just "seppy" or "cee-pee." One pilot group mentioned that waypoints with numbers in the name (e.g., "KA270") were especially difficult to use, which agrees with findings from [13].
Difficulties that participants encountered were categorized as being related to the IFP, to the chart, or both. If a chart manufacturer could independently resolve the issue by adjusting the graphics, that was considered to be a chartspecific issue. Anything that required a change to the IFP design or content that the chart producer obtained from the IFP designers was considered to be induced by the IFP design. Some difficulties arise from the IFP design parameters; these can only be resolved by IFP designers. Fig. 1, an IAP into Scottsdale, Arizona, shows an example of a difficulty created when translating the IFP design into a visual chart. This IAP chart uses insets in the plan view to show a path that is too long to fit in the normal scaled area. The insets display the path in a "non-contiguous" manner, which can be difficult to follow. The KORRY arrival used in the study also used an inset to show a non-contiguous path. The Denver and Boise RNAV (RNP) AR procedures also had to use non-contiguous paths. A third, relatively small, source of complexity is the chart composition itself, including the arrangement of the sections or elements within the visual design. For example, notes are scattered about on the KORRY arrival, and one note, about a maximum holding speed, was considered important, but was sometimes difficult to find.

D. Line Pilot Subjective Complexity Framework
The subjective complexity framework for line pilots (Fig. 2) summarizes the main results of our study. The gears in the center of Fig. 2 show the three types of issues and how they drive each other. Individual sources of complexity are listed in each of the boxes next to a gear. (In the full report, we provide examples for each one of the individual sources of subjective complexity.) Operational complexity factors are shown in a cloud in the upper left corner. While operational complexity affects final implementation, it is not connected directly to any part of the IFP and chart design. Two cross-cutting issues are listed in the box on the left of the diagram, visual density, and inconsistencies between different IFPs. IFP design parameters are the primary source of subjective complexity, as indicated by the largest gear. IFP design parameters can produce paths that are difficult to draw in a standard chart format, which can result in charts that need to be reviewed more carefully by pilots. Chart specific issues (e.g., placement of individual notes) can also create some confusion for the pilot during the review and/or briefing.    IFP design parameters include, for example, the number of transitions, constraints, waypoint names, and the number and content of notes. All of these are provided to chart designers in the IFP specification. The chart designers have to deal with how to depict all the IFP elements, including all the transitions, holds, constraints, notes, etc. Sometimes the designer is unable to draw the IFP flight path in the normal chart format, and is therefore required to consider alternative formats such as visually non-contiguous paths (which include multiple-page formats) or larger page sizes for paper charts. Finally, novel chart formats create situations where individuals make one-off decisions about chart composition (e.g., placement of elements and arrangement of data). There can be inconsistencies because of the novelty of some chart formats.
There are two cross-cutting issues that go beyond the IFP design and the chart design. First, if an airspace is dense (with a lot of air traffic, standard routes, or both), that necessarily creates a complex IFP design, with corresponding visual complexity from the density of waypoints and flight paths. Second, there may be issues regarding the consistency between different types of IFPs. As an example, consider Fig. 3 and Fig.  4, which are excerpts from an RNP AR IAP and RNAV STAR, respectively. In Fig. 3, altitudes are shown along flight path segments, but in Fig. 4, altitudes are shown at waypoints along the path, not along the segment. Four pilot groups commented that it was odd to see altitudes along segments instead of at individual waypoints along the IAP. As the numbers of constraints and path segments increase on SIDs and STARs, there may be more parallels with IAPs, which may uncover latent discrepancies between these different IFPs.

V. RECOMMENDATIONS
Results of this study provide insights for all stakeholders in the process of IFP design and implementation. Stakeholders will need to develop actions to address these recommendations cooperatively. Each of the recommendations below originates from a pilot need. There may be different ways of satisfying these pilot needs, so we do not attempt to provide prescriptive solutions.

A. IFP Design Recommendations
• Minimize path constraints. Constraints create pilot workload during reviews and briefings. They must be reviewed if the route is amended. Pilots have to actively manage and monitor constraints in flight. Workload of managing constraints can vary greatly depending on the aircraft equipment and its ease of use.
• Minimize flight path transitions. Transitions add variability to the flight path, they add visual complexity to charts, and they add a decision point for pilots.
• Ensure that energy profiles are smooth between adjoining IFPs and/or segments of IFPs. Pilots manage and monitor aircraft energy as they climb and descend. The flight path should allow a smooth climb and descent, without sudden changes that surprise the pilot.
• Waypoint names should be pronounceable, short (with few syllables), and, ideally, familiar. Pilots review waypoints in their crew briefings, which are quick and focused. Awkward waypoint names take extra time and may create confusion.
• Minimize and prioritize notes. Be aware of the intended audience and write the note for that audience. Pilots learn to ignore notes if they do not apply.

B. Recommendations Related to Operational Complexity
• IFP designers should assume that one or more operational complexity factors will be a factor in normal operations. The IFP should be designed to absorb normal operational variations. Do not assume best case conditions for normal operations.
• IFP designers should be better informed about aircraft equipment variation and flight deck tasks and perspectives. Designers should understand how flight crews manage flight path constraints and air traffic interventions in particular.
• The range within which runway changes are not allowed should be increased to reduce flight crew workload during arrivals, especially for arrivals into terminal areas with high traffic density or complex arrival routes. This particular topic is also addressed in [14]. It has been actively debated because this issue directly trades off flexibility and complexity; Allowing later runway changes increases flexibility for airspace operations, but increases complexity for flight crews.

C. Joint IFP Design and Chart Recommendations
• Develop guidance to decide when to separate flight paths into different IFPs or keep a single IFP. If many paths are on a single IFP, there should be additional guidance on whether to split the depiction onto multiple chart pages (or electronic images). For example, the initial Boise RNAV (RNP) AR approaches were eventually split into separate IFPs because they were trying to combine features of both an arrival and an approach into a single IFP.
• Clarify and separate notes based on purpose. Determine whether the note is for action or awareness, and consider whether the two types could be separated for pilots. Notes for action are more important to flight crews than notes for awareness.
• Reduce the overall number of notes. Determine whether the chart is the best means for conveying specific notes or if another location or method of communication would be better. Determine whether some notes are no longer useful and remove these.
• Provide data on SIDs and STARs to give pilots a general sense of the terrain in the terminal area. Pilots could use this data to judge whether or not terrain will be a factor that they should plan for more carefully. Providing data directly for this purpose will discourage pilots from using potentially incorrect information from approach procedure minimum safe altitudes.

D. Chart Recommendations
• Clearly indicate the "top altitude" on SIDs. Pilots know that this altitude is important for "climb via" clearances. They should not spend time hunting for it. Pilots need to enter this altitude into their autoflight system.
• Carefully place notes. Consider grouping notes to make them easier to find.
• Have a clear graphical connection between sections if the flight path is split between chart sections (or pages/images). If the flight path is split and the graphical connections are not obvious, pilots may misunderstand the path or miss parts of it.
• Emphasize information that is non-standard and of operational importance (e.g., non-standard altitude and/or speed constraints). Pilots brief non-standard aspects of the IFP so that they know what to expect.

E. Joint IFP Design and Training Recommendations
• Identify and eliminate ambiguity through training and clearer IFP design principles. Assume that not all pilots will have a detailed technical understanding of the IFP design or intention, or how it might unfold on their aircraft. For example, some pilots may not be able to predict in advance how their VNAV system will handle particular constraints.
• Improve pilot training and IFP design principles to improve pilot understanding on why IFPs are designed the way they are. This will improve pilot's ability to plan for different types of IFP, and will improve their resiliency to operational variations.

VI. SUMMARY OF KEY POINTS
We have made significant progress in understanding line pilot perspectives on the complexity of IFPs through this study. Our findings can be applied to any type of IFP, PBN or conventional. It appears, however, that more PBN IFPs use features (e.g., multiple transitions and altitude constraints) that are associated with complexity for pilots.
One of the key contributions of this work was to define and separate different types of complexity. We now have a language for discussing different types of complexity that are often confused: operational complexity, IFP design complexity, chart complexity, subjective complexity, visual complexity, objective complexity. We hope that stakeholders in IFP development will see the value of these distinctions in understanding how to simplify IFPs that are difficult to use in real operations. Identifying the type of complexity will help to identify which stakeholder(s) can address that issue.
A second key contribution was to identify a more comprehensive list of subjective complexity factors. The list of factors was longer than we initially anticipated. We went beyond the original goal of just gathering a list, to developing a framework for subjective complexity. This framework is pilotcentered instead of designer-centered. It illustrates how different types of complexity are related, and how they drive the final product (both the IFP and chart design). Our framework may help those involved in IFP design and implementation to anticipate how the IFP might be perceived and used by pilots before flight operations begin.
A third key contribution was to identify operational complexity as a separate problem area. This issue cannot be addressed by IFP designers alone. To understand operational complexity further, we will need to understand variables that were out of scope for this study, such as pilot training, Air Traffic controller training, and Air Traffic priorities, requirements, and tradeoffs.
Another important product of this study is a better understanding of how pilots use charts during reviews and briefings. Pilots use charts for both tasks, but in different ways. This is the first study we know of that explicitly looked at how pilots use SID and STAR IFPs. Previous work focused on the design of IAPs (e.g., [15]). Much of the critical work for understanding and flying the IFP is done during the IFP review and entry. Chart manufacturers may find this helpful for improving their designs, and for developing future data-driven chart products. We also observed that good IFP reviews go hand-in-hand with good briefings. Our findings may be useful for operators to consider in their pilot training.
We also heard from pilots that they want to understand more about why IFPs are designed the way they are. Understanding the logic of IFP design may help them to review and fly the IFP better. For example, our participants wanted to know why there were so many constraints in some IFPs. They wanted to know the purpose of a constraint, and whether they could get relief from ATC by request. Finally they wanted to be sure that non-standard constraints were salient. Interestingly, clarifying these points to pilots may also help IFP designers to be more consistent in their use of constraints.
Finally, we provide recommendations that may help ease operational implementation. These recommendations should be considered by all stakeholders cooperatively to develop solutions that will positively impact flight operations.

VII. PLANS
A full report on this study is in preparation. We also plan to continue briefing industry and government organizations, as requested, to increase awareness among aviation professionals in government and industry.
We hope to expand this effort at some point to understand the needs and complexities of PBN IFPs from the Air Traffic perspective, to create a parallel model of complexity from their point of view. Another direction we are considering is how we can measure the performance effects of subjective complexity. Our participants used many examples to illustrate how they handled difficult instrument flight procedures and it would be useful to observe how flight crews actually handle these in a simulator. We are also interested to understand how pilot training on handling PBN IFPs could be improved.