Ethical mission definition and execution for maritime robotic vehicles: A practical approach

Many types of robotic vehicles are increasingly utilized in both civilian and military maritime missions. Some amount of human supervision is typically present in such operations, thereby ensuring appropriate accountability in case of mission accidents or errors. However, there is growing interest in augmenting the degree of independence of such vehicles, up to and including full autonomy. A primary challenge in the face of reduced operator oversight is to maintain full human responsibility for ethical robot behavior.


I. INTRODUCTION
Many experts and practitioners have worked long and hard towards achieving functionally capable robots. Unfortunately, progress in ethical control of unmanned systems has been elusive and problematic. Common paradigms that assume an always-amoral robot or that require undemonstrated moralitybased artificial intelligence (AI) are equally untenable. For better or worse, actors around the world are rapidly designing and deploying mobile unmanned systems to augment human capabilities. Thus theory must meet practice. This paper adapts policies and procedures for ethical responsibility and authority that have been proven to work effectively in collaborative military operations. Since ethical responsibility is not limited to military weapons but can also apply to even routine task completion, this approach appears to have broad usefulness for civil application of unmanned systems as well.
The authors' experience across four decades of robotic and military operations has demonstrated that robot mission tasks and goals can be clearly defined and refined with corresponding degrees of internal control supervision. Adding well-specified constraints can supplement mission orders, providing an ethical basis for unmanned system tasking that matches human understanding of similar responsibilities. Careful structuring of mission orders in the form of a mathematical construct called a Mission Execution Automaton (MEA) demonstrates a theoretically sound and scalable basis to this approach. Further, a Mission Execution Ontology based on principles of description logics provides mathematical assurances that mission definitions are semantically complete, including ethical constraints whenever appropriate.
The advent of digital computing, the emergence of AI, and the incorporation of both into a variety of robotic devices have brought ethical concerns to the forefront of debate. It has become quite apparent that legal and moral responsibility cannot be addressed by a set of fixed "laws" intended to constrain robot behavior. Reasoning about abstract situations from high-level principles [1] is beyond current capabilities to describe (much less arbitrate), and such essential imperatives cannot be regulated into irrelevance. Nevertheless, a number of important observations inform this work: • Robots do what programmers and operators tell them to do-not what programmers and operators mean to tell them to do.
• Apparent intelligence notwithstanding, a robot is an inanimate object and cannot assume moral or legal responsibility for an action's consequences.
• For robot ethics to bear any tangible meaning, ultimate accountability must reside with the human programmers, manufacturers, and operators.
• Legal and moral liability requires that involved parties are in a position to reasonably foresee the outcomes for which they are being held responsible.
These observations are fairly widely accepted, but nevertheless can still lead ethicists to different conclusions. In debating military use of autonomous systems, for instance, Rob Sparrow of the International Committee for Robot Arms Control applies Jus en Bello requirements to argue that the military use of lethal robots is inherently unethical because robots cannot be held accountable for their actions [2]. Ronald Arkin, on the other hand, accepts the premises of Sparrow's argument but comes to the opposite conclusion-that if an autonomous system is capable of making a lethal decision more reliably than a human, then it is inherently unethical to not use that system [3]. This work demonstrates that such observations can inform a broader framework for ethical operation of intelligent robots, one that is realizable with current technologies and guided by human ethical decision making, without any requirement for black-box artificial intelligence that inevitably leads to second-guessing.
II. MILITARY OPERATIONS AS AN ANALOGY Authority and responsibility in military operations provide a useful analogy. Military commanders are provided forces over which they exercise control, are assigned missions that they are expected to accomplish, and are held responsible for the proper employment of all assigned assets. More recently, militaries have relied on increasingly automated systems, however, automation does not obviate the commander's responsibility. Ultimately, it does not matter whether a military leader is employing a system of people or a system of machines: authority implies responsibility [4]. Accountability in military operations requires a level of trust that is based on a number of important factors. First, subordinate units must be qualified for the designated task requirements. Second, mission orders must be unambiguous and fully understood by tasked units. Finally, subordinate units must accurately assess task progress during execution. This trust relationship provides assurance that properly employed subordinates do not impose undue risk. Improperly employed subordinates, on the other hand, do impose undue risk for which the commander is rightfully held responsible.
A variation of this ethical mechanism can be applied to robots in both military and civilian applications as a corollary to the well-established legal principle of vicarious liability [5]. That is, operators of autonomous and unmanned systems can be held responsible for undesirable outcomes that they are in a position to prevent.
All robots possess a finite set of operational and sensory capabilities, the complexity of which varies from robot to robot. It is reasonable, then, to trust a robot to execute those atomic capabilities. It follows that an operator can assume moral and legal liability for missions comprised of these trusted capabilities [6]. For this framework to be practically feasible, robot operators must be provided adequate mission assurance and understanding to assume responsibility. This can be achieved by meeting three requirements: • Robot missions must be defined in a mathematically sound manner that ensures that the mission will progress as intended in all circumstances.
• There must be no means by which an approved mission can be semantically modified between approval and execution by the target vehicle.
• Mission tasking and associated constraints must be comprised entirely of trusted atomic vehicle-specific behaviors, and the vehicle must be able to continually evaluate both behavior and constraint status at run time.

CONSTRAINTS
In describing complex tasks, humans often divide them into series of subordinate tasks to be executed in order. For instance, a complex task during which a manned vehicle is to conduct searches and collect environmental samples before rendezvousing with another vehicle might be specified as depicted in Fig. 1. Providing the vehicle's operator understands the individual subtasks, the mission can be reliably executed.
The vehicle operator implicitly relies on a discrete decision process to periodically take stock of the situation, determine the current task's status, and proceed to the next task when appropriate. This decision process is commonly referred to as an Observe-Orient-Decide-Act (OODA) loop when referring to military and other human operations, or as a Sense-Decide-Act (SDA) loop when referring to autonomous agent activities [7] [8].
One aspect of the above mission must be accounted for, however, to support execution by an autonomous agent: the implicit assumption of success for each task. When facing task failure, a human is able to use best judgement or request guidance from higher authority. This is not necessarily an option for autonomous agents. Rather, the appropriate course of action must be included in the mission description. This can be achieved by specifying subsequent alternative tasks that execute in the event of any particular mission task's success or failure. Branching based on task success or failure results in missions whose execution is characterized by a sequence of successful and unsuccessful task executions. It is appropriate, then, to refer to the individual tasks as goals to be achieved rather than simply as tasks. A possible modification of the mission from Fig. 1 is provided in Fig. 2. Interestingly, the SDA/OODA decision loop is still suitable for execution control of this revised mission. Fig. 2 gives rise to a graphical representation of the naturallanguage mission definition. The flow graph of Fig. 3 is one among many potential representational forms for this and many other missions. It is of particular interest because it provides an intuitive depiction of a potentially complex mission. In fact, this mission specification can be used to mentally "rehearse" the mission by intentionally traversing the graph from start to finish while testing success and failure branches at every step.
While not yet providing mathematical rigor, this ability to informally traverse task sequences is an important step towards providing assurance to the responsible operator that the mission will progress according to human intent.
As presented so far, this mission definition paradigm does not explicitly address the issue of ethical mission execution. Specifically, no mechanism has been suggested at this stage to define ethical constraints affecting the overall mission or individual tasks. For instance, it is apparent that an unmanned underwater vehicle (UUV) with an appropriate search behavior can achieve goals 1 and 3 of the example mission. Unfortunately, it may or may not be able to do so while avoiding detection, remaining clear of other vehicles, or maintaining a specific navigational accuracy. If any of these or any other ethical constraints are to be applied to the mission, then they must be incorporated into the mission specification. Further, from the standpoint of operator accountability, constraints must be specified in a manner that preserves the ability to trace high-level mission flow and in a way that is enforceable by the target vehicle.
An operator may determine that certain constraints must be enforced from launch until recovery (e.g., all safety systems must remain operational), while others only need to be enforced during the execution of specific goals (e.g., maintaining safety depth in the search area). As an example, mission-and goal-level constraints might be applied to the example UUV mission as depicted in Fig. 4. This construct still supports the prior rehearsal of missions and also allows for the in situ consideration of whole-mission and goal-specific constraints [9].
Under the binary branching model of Fig. 3, an impending ethical constraint violation implicitly equates to goal failure. This might be acceptable, but it might be desirable to treat impending constraint violations differently than simple failure. A third potential goal-execution outcome and a corresponding branching option in the mission flow structure might be useful. That is, execution of an individual goal terminates upon goal success, goal failure, or impending violation of a constraint applied to that goal. A modification of the example mission to implement this ternary branching model is graphically depicted in Fig. 5.
The flow graph mission specification described here is declarative. At this level of abstraction, individual goals execute sequentially according to the mission graph irrespective of time, and each goal predictably terminates in one of three possible states. This approach eliminates any need to make assumptions or guesses concerning intended vehicle Mission-flow graph for search and sample mission, human or autonomous agent execution [19].     [12] and has been utilized extensively with the Rational Behavior Model (RBM) [13]. RBM organizes robot control requirements into strategic, tactical, and execution levels. The strategic level controls high-level mission flow, accomplishing high-level tasks by issuing commands to the tactical level. The tactical level executes self-contained behaviors in response to strategic-level commands, notifying the strategic level of command outcomes (success, failure, or constraint violation). The execution level provides real-time control of actuators and sensors as directed by the tactical level. Evidently, the strategic level is well suited to execute mission-flow diagrams, and tactical-level behaviors can be comprised of atomic capabilities of a particular target vehicle.
If properly encoded, the strategic-level mission-flow diagram forms an executable mission specification. This human-and-machine compatible form is in line with ethical framework requirements and provides for human-based testing of mission code prior to robot execution. Further, from the perspective of the strategic level, it does not matter whether the tactical level behavior is executed by an actual robot, a computational model, or a human being, making full-fidelity mission-flow-diagram rehearsal possible.

B. Mission Execution Automaton (MEA) Definition
Mathematical rigor of strategic-level execution steps is obtained by observing that the run-time traversal of the mission-flow diagram is similar to the operation of a mathematical formalism called a Turing Machine (TM) [14]. TMs have a number of fundamental properties that are particularly important in the field of computer science [14] [15] but are generally considered impractical for real-world utilization [16]. They do, however, provide a strong theoretical foundation upon which to build a mathematically sound strategic-level mission-flow diagram execution mechanism suitable for both robots and human operators. With some modification, TM execution mechanics can provide an execution engine for mission-flow diagrams expressed as finite state machines (FSM) [14] [15]. More specifically, the mission-flow diagrams can be viewed as robot programs that are executable using TM semantics. Relaxation of some additional TM formalisms yields a mathematical construct that subsumes the more constrained TM [16].
This TM generalization is referred to here as a Mission Execution Automaton (MEA) which consists of a Mission Execution Engine (MEE) and an arbitrary set of mission orders in FSM form, and a communication link to at least one human or robot external agent capable of carrying out orders from a given finite set (as issued by the MEE) and returning results from another finite set [17] [18].

C. Strategic-Level Mission Rehearsal and Testing
MEA implementations were initially developed in Lisp and Prolog [19] [16] [18]. The declarative nature of Prolog in  particular aligns well with the mission-flow diagram and makes Prolog mission orders intuitively understandable by non-programmers. Strengths notwithstanding, it is important to note that there is nothing inherently unique about Prolog, and the MEE and mission orders can be accurately created with any Turing-complete computer language [20] [9] [21].
A similarly capable, independent implementation uses the Hierarchical Task Network (HTN) behavior model and Python programming language in the Combined Arms Analysis Tool for the 21st Century (COMBATXXI), a simulation tool developed and used by the U.S. Army and U.S. Marine Corps within various analytic studies [22] [23]. Like the earlier Prolog simulation, the COMBATXXI simulation allows testing of the Strategic Level mission flow by a human operator, with typical results depicted in Fig. 6. In the simulation, the Strategic Level orders commencement of individual goals and the human operator reports success, failure, or constraint-based termination of each goal. The depicted traces in Fig. 6 correspond to instances where each goal terminates due to potential constraint violation and where each goal completes successfully.
This ability to formally test a mission definition by allowing a human operator to assume the role of the tactical level provides assurances that the branching upon individual goal success and failure matches the operator's intent. The previously discussed premise that ethical operation requires formal assurance that the mission will proceed as intended in all circumstances is thus satisfied so long as mission-flow graph size and structure is constrained so that it is exhaustively testable.
Based on this experience it is reasonable to conclude that flow graphs represent a higher level of abstraction for mission specification than any unconstrained text-based coding language. Moreover, advances in graphical coding [24] may eventually allow non-programmers to completely specify robot missions by constructing a flow graph such as Fig. 5 directly on a computer screen. Such advances can further enhance the comprehension, supervision and accountability of mission experts for producing legally valid mission definitions.

D. Progressive Refinement of Complex Mission Tasks
Referring to Fig. 5, it is apparent that Goal 1 is achievable only if the person or software at the tactical level has considerable knowledge about Area A and how to search it. Stated differently, it can be completed only if a tactical-level behavior can be invoked that will execute the search appropriately. To make this concrete, suppose that Area A contains hazards that are potentially harmful to (or impassible by) the search vehicle and that no current map of the area of interest is available. A classic algorithm for exploring for such circumstances is depth first search [25]. Such a search first discretizes the search area into a finite set of cells and proceeds by systematically moving the vehicle between cells. At each step, the search tests the currently occupied cell to see if the search object is there (success). If it is not, the search proceeds by moving the vehicle into a previously unexplored, adjacent cell. If no such cell exists, then the vehicle retreats to the previous cell and the process continues. If the vehicle finds itself at the starting location with no adjacent unexplored cells, then the search is complete (failure). Fig. 7 depicts a flow graph for the above-described process, and further represents a refinement of Goal 1 of Fig. 5. If trusted vehicle behaviors corresponding to each of the figure's goals exist, they might be commanded by a human operator acting on behalf of the tactical level (example available in [18]). Alternatively, this depiction may be understood as a tactical level implementation capable of autonomous execution of Goal 1. Note that all applicable constraints must be continuously observed throughout execution.

Ethical Mission Execution Trace
This task decomposition raises an important implementation question: does Fig. 7 accurately represent the semantics of the desired behavior? Interestingly, since the flow diagram contains several decision loops, answering that particular question cannot be confirmed by exhaustive testing as was possible for the strategic-level mission orders. Such test limitations occur because, in general, there is no guarantee that such a search will always terminate for arbitrary terrain, targets and obstacles. Fortunately it is known that for a finite search area, depth first search will eventually terminate with either success or failure in searching for a specified goal [25]. Nevertheless, exhaustive testing for all possible terrain samples is not possible. Instead, the most that can be asked for is to show that all phase transitions in the given flow graph are correct [18].
It turns out that the inability to exhaustively prove correctness of a tactical-level flow graph is not as serious as it might at first appear. This is because tactical-level algorithm failure is no different than other outcomes that might cause a strategic-level goal to fail. Since strategic-level goal failure is accounted for in the overall mission-flow graph, the mission can still continue as planned. To make absolutely sure that such dependability occurs, a time-out condition resulting in goal failure must be incorporated at the tactical level [13].
Regardless, manual execution of progressively refined behaviors defined as flow graphs provides a mechanism for extending existing tactical-level behaviors. That is, once a flow graph accomplishing a specific purpose (e.g., depth-first search) has been suitably vetted, it effectively becomes a trusted tactical-level behavior itself. This means reinterpreting tactical-level flow graphs as code specifications or templates rather than actual code to be executed [18].

A. Phoenix Autonomous Underwater Vehicle (AUV)
Up to this point, presented results have related to the strategic level and the tactical level of RBM software for a single example of a "search and sample" mission for a notional UUV with results obtained through simulation. However, beginning in 1993, in parallel with formalization and publication of details of RBM [13], the value and practicality of this approach was demonstrated through a series of openocean experiments involving two small unmanned submarines.
The Phoenix AUV was an unmanned submarine designed and built at the Naval Postgraduate School (NPS) beginning in 1990. Weighing approximately 500 pounds, Phoenix included four cross-body thrusters, enabling active control of five degrees of motion (x, y, z, pitch and yaw) [26].
Large numbers of high-fidelity physics-based simulations were needed to correctly develop and test what were then considered AI approaches to replace human supervision. While this simulation involved distinct mission phases in the form of a command "script", similar to Fig. 1, no binary flow graph with phase failure contingencies was abstracted from these phases making exhaustive testing (and thus proof of correctness) of a mission impossible. Moreover, no concept of ethically constrained behavior was attempted in any of this work.

B. Aries AUV
Lessons learned from the Phoenix UUV were incorporated into the second-generation NPS UUV, the Aries. Specifically designed for open-ocean surveys, Aries was a somewhat larger vehicle that utilized more efficient forward thrusters but lacked cross-body thrusters [20].
An extensive and accurate physically based model of the vehicle and its environment, with three-dimensional (3D) graphical display, the Autonomous Unmanned Vehicle (AUV) Workbench, was developed and used for real-time testing of robot mission software [9] [21] [27].
Aries AUV missions were defined with the NPS-developed Autonomous Vehicle Command Language (AVCL), a schemaconstrained XML data model supporting autonomous vehicle mission implementation, execution, and management [20]. While the mathematical concept of an MEA had not been developed at the time of AVCL's development, a fixed set of goal types is provided and branching upon goal completion is supported as well. Thus, AVCL is suitable for the realization of mission flow diagrams. Further, AVCL was intentionally designed to support implementation of RBM strategic and tactical levels and was utilized to define RBM-controlled Aries missions for AUV Workbench simulation and real-world openocean tests.
Simulation of a mission consisting of an AVCL specification for a search goal similar to Goal 1 of Fig. 5, while steering clear of avoidance areas specified as constraints in the AUV Workbench, is shown in Fig. 8. During the mission, the tactical level plans a path and maneuvers to the search area

A. Description Logics (DL) and a Robot Mission Ontology
Thus far, this discussion has focused on providing robot operators the ability to rigorously define and test strategic-level missions. The premises upon which this proposed framework for ethical robot tasking rests also require that an actual target vehicle can execute these missions without further translation. Fortunately, mathematical logic provides a mechanism for bridging strategic level missions and vehicle-specific code for specifying and ordering tactical-level behaviors.
Description Logics (DL) are a mathematical family of logic-based knowledge representation systems for describing concepts and roles within a system. DL ontologies can define the relationships in a system, how they operate, how they are used, and to what specific entities they apply. DLs are carefully and strictly defined to enable computationally efficient reasoning that can identify hidden relationships and errors, such as rule violations or contradictions [28].
DLs provide the foundation of the Semantic Web, a set of standards-based extensions to the World Wide Web that leverage DLs' expressiveness and rigor to provide extensive knowledge representation, discovery, and utilization capabilities [29]. Notably, the Web Ontology Language (OWL) [30] encodes a particularly powerful DL in a validatable plain-text computer-readable form [31]. OWL is used here to define a robot mission description and execution ontology that applies and enforces MEA semantics.

B. Mission Execution Ontology (MEO)
A Mission Execution Ontology (MEO) serves a number of purposes. First, it enables a formal and semantically rich description of the characteristics of a MEA mission description. For instance, OWL expressions are used to declare the existence of concepts such as "Mission", "Goal", and "Constraint" and also to define possible relationships between concepts. As an example, a "Mission" entity must have an "includes" relationship with at least one "Goal" entity and must have a "startsWith" relationship with exactly one of those entities. A partial graphical depiction of the concepts and relationships defined in the MEO is provided in Fig. 9.
In addition to the concepts abstracted directly from mission-flow diagram semantics, the MEO introduces the "Vehicle" concept providing for the inclusion of specific target vehicles in the mission-planning process. The "canExecute" and "canIdentify" relationships allow mission planners to explicitly assert that the intended target vehicle has a tacticallevel behavior capable of completing a goal and recognizing potential violation of a constraint, respectively. Using this concept and relationship semantics enforced by MEO rules, it is impossible to define a valid mission that cannot be executed by the intended vehicle. In addition to describing the "Mission", "Goal", "Constraint", and "Vehicle" concepts and how they relate to one another, the MEO allows the application of those concepts to real-world entities and the establishment of relationships among these entities. This means that the atomic entities to which the "Goal" and "Constraint" concepts are applied will be executable by specific target vehicles. The MEO can be applied to a snippet of vehicle-specific executable code by defining an OWL statement declaring its existence. Additional OWL statements can then declaratively apply concepts and establish relationships. Fig. 10 shows the mission of Fig. 5 expressed using OWL according to the MEO of Fig. 9, produced from RDF/OWL source using the Protégé Ontology Editor [32]. Detailed comparison of Fig. 5 with Fig. 10 (which was automatically generated) shows that the intended mission has been correctly encoded, is validated, and is executable by the target vehicle. Based on inspection of each figure, the human mission commander can confirm that all necessary ethical constraints have been applied in the correct contexts.
The ability to provide a full description of all goals and constraints using vehicle-executable code further strengthens the MEA construct. Specifically, not only is it impossible to define a mission for a particular vehicle without explicit "canExecute" relationships between the vehicle and all mission goals and "canIdentify" relationships between the vehicle and all mission constraints, but it is also impossible to assert these relationships without an appropriate vehicle-specific encoding of all mission goals and constraints.
Automated reasoning is an important tool for ensuring strategic-level mission validity before conducting exhaustive testing described in previous sections. If, for instance, a mission includes goals that are not executable by the target vehicle, a reasoner can quickly identify this shortcoming. Similarly a reasoner can detect mission flow-graph structural errors based on ontology rules, thereby precluding illogical loops, unreachable goals, and orphan goals without specified successors. A reasoner can also simplify mission definition by detecting implicit relationships that are not explicitly specified.
As described, a DL-based mission execution ontology defined in OWL ties MEA semantics discussed in previous sections to actual target vehicles. The ontology ensures not only structural validity of a mission-flow graph, but also its executability on the target vehicle.
Thus, all of the requirements originally posed for assignment of human responsibility -that the mission is defined in a mathematically rigorous manner, that the mission is understandable by both the human operator and the target vehicle, and that the mission is comprised entirely of trusted vehicle behaviors -are captured in the human-approved mission orders and enforced by the strict semantics of the MEO. The ability to validate ethical correctness and completeness is an important new addition to cooperative mission definition and execution between humans and robots.

VII. SUMMARY AND CONCLUSIONS
Ethical operation of robotic systems requires human accountability. In both the legal and moral sense, this implies that human operators be in a position to understand, and therefore control, robot mission outcomes. This level of understanding can be achieved through the satisfaction of three requirements: operator understanding of high-level mission flow, mission descriptions understandable to both human operators and target vehicles, and mission descriptions consisting entirely of trusted behaviors and constraints.
Early NPS UUV missions were executed as a result of an inferencing process over a mission definition comprised of a set of rules and facts. This approach provided no means of proving the correctness of strategic-level missions comparable to the exhaustive testing of MEA flow graphs. For this type of system, errors in the mission axiom set can lead to unpredictable and potentially hazardous or self-defeating system execution behavior. Ultimately, this unpredictability precludes the formal assumption of responsibility or liability for robot missions.
Further, AI approaches in general almost invariably make use of easily confounded inferential reasoning or statistical pattern recognition. Applying such broad abstractions to the innumerable situations that can arise in the real world is inherently unpredictable, and also makes unrealistic any assumption of responsibility by human operators. It is therefore apparent that the abstract reasoning of general AI approaches is inappropriate, at least at the present time, for the highest level of robot mission definition and control.
Algorithms cannot replace human responsibility. Even so, a fully testable technology (such as that provided by the MEA and MEO formalisms) allows for the assignment of human accountability.
Specifically, the MEA provides a mathematically rigorous mechanism for mission definition and execution as an exhaustively testable flow diagram. This approach ensures that accountable operators can fully understand all high-level task sequences before authorizing robot operations. The MEO employs DLs and Semantic Web technologies to provide strong assurances that MEA mission definitions are semantically correct and fully executable by specific target vehicles.
By applying the best strengths of human ethical responsibility, repeatable formal logic and directable unmanned systems together, these capabilities provide a practical framework for ethically grounded human supervision of unmanned systems.