Bringing Trust to Autonomous Mobility

Last decade has been characterized by a huge advancement in the field of automated and connected transport. However, fully autonomous systems still need a lot of effort in order to be applied in transportation. Meanwhile, mixed traffic environments with semi-autonomous vehicles is becoming a norm. In such conditions, vehicles are passing the dynamic driving task back to the human by sending to drivers Requests to Intervene (RtI). At the same time, there is a need to evolve driver’s training in order to be able to safely use semi-automated vehicles, whereas driver intervention performance has to be made an integral part of both driver and technology assessment. Furthermore, the ethical implications of automated decision-making need to be properly assessed, giving rise to novel risk and liability analysis models. In this conceptual paper we present our vision to maximise the safety, trust and acceptance of automated vehicles. To achieve that, we propose an assessment framework to evaluate different technologies involved in Automated Driving Systems (ADS).


I. INTRODUCTION
This decade has brought into our daily lives autonomous vehicles [1], while most of the well-known automakers have already began executing their plans to commercially release autonomous vehicles by 2020-2021 [2]. However, current projections of market analysts, including Blackrock [3] and UBS [4], indicate that broad adoption of fully autonomous vehicles might be decades away. This in turn suggests that the human factor will remain essential for the safety and performance of road transport in the forthcoming decades, mainly for two reasons: a) due to the necessary driver-vehicle interaction in cases where the boundaries of the Operational Design Domain (ODD) of an Automated Driving System (ADS) are being reached, and b) because of the co-existence of fully-, semi-and non-autonomous vehicles, which is likely to be raising unexpected challenges.
Central to the human role in the Connected Automated Driving (CAD) is the transition from automated to manual driving mode. This might be system-initiated, whereby the ADS issue a Request to Intervene (RtI), i.e. notifies the human driver that he should promptly take-over control and perform the Dynamic Driving Task (DDT) fallback [5]. This can happen when the ADS detects a system limit, e.g. because of sensor malfunction, extreme weather conditions, appearance of evolving accident scenes, unexpected road blocks, hazardous traffic code violation from another vehicle, falling of goods, etc. However, the transition can also be userinitiated, e.g. to provide a corridor for emergency vehicle access, or follow hand signals given by a traffic enforcement officer [6].
Evidently, in such a dynamic driver-vehicle interaction scheme, several challenges arise. First, in parallel to the detection of system limits, the driver's availability to intervene has to be evaluated, through continuous Driver State Monitoring (DSM). Second, the transition's success has to be ensured by proactively allowing sufficient lead time and utilising appropriate and comprehensible Human-Machine Interfaces (HMIs) that maximise situation awareness and intervention performance. Third, driver training has to evolve to meet the safety challenges of "driving" an automated vehicle. Fourth, measuring of the driver intervention performance as well as of ADS user acceptance, depending on different levels of automation, take-over requests, etc., becomes essential. Moreover, the implications of automated decision-making from a legal or ethics perspective have to be examined, and risk models (e.g. addressing liability issues) for the co-existence of various automation levels have to be developed. Notably, there is a lack of standards, pilot results and established practices in the aforementioned fields, such as for HMIs in automated driving and for takeover performance assessment. Running parallel to these challenges is the dimension of trust: not interpersonal trust but trust in technology and, specifically, in automation.
In this paper we present the vision of Trustonomy (a neologism from the combination of trust and autonomy) which is to raise the safety, trust and acceptance of automated vehicles by helping to address the aforementioned challenges through a well-integrated and inter-disciplinary approach. The rest of the paper is organised as follows: in Section II we present the Trustonomy objectives and related work; Section III includes our proposed approach; the system architecture is presented in Section IV, while Section V concludes the paper.
II. TRUSTONOMY OBJECTIVES AND RELATED WORK In order to address the challenges mentioned in the previous section, we have identified six specific objectives which are depicted in Fig. 1.
A more detailed description of the envisioned objectives is presented below: • Develop a Methodological Framework for the operational assessment of different DSM systems: DSM plays a crucial role especially for L3 vehicles, in which humans are in the loop, being involved in driving operations. The most critical scenario is represented by the RtI, in which the human driver has to take control of the vehicle. The DSM has to establish the driver status and his ability to safely accomplish it. Relevant research [7] has pointed out that, when humans do not pay attention at all when the vehicle is driving itself, they could not shift attention quickly enough to safely take control of the vehicle. Continuous monitoring of the driver is a possible solution able to mitigate this problem, adapting the RtI procedures, in particular during the takeover phase. The monitoring can be achieved by various methodologies, such us by monitoring the eye movement of the driver [8] or by monitoring driver's blood pressure [9]. The DSM can trigger the appropriate notifications and warnings to the driver, when he decides to resume the manual control. In [10] the researchers agree that in the next few years, there will be manual driven vehicles with several autonomous features requiring a short notice intervention of the driver, therefore, a DSM system is necessary to support time-efficient transition of control.
• Develop a Methodological Framework for the operational assessment of various HMI designs: There is still a lack of understanding regarding methods to evaluate HMI in CAD vehicles. One perspective argues that CAD vehicles could have an HMI design similar to the one used in conventional (L0-L2) vehicles. These systems only help the driver to make adequate decisions, and the driver is still responsible for the decisions he made. Even though L3 systems still require full-time supervision of the driver, the HMI must limit the effects of driver periodical inactivity or driver's fatigue. L4 systems, which allow to drive the vehicle mostly in automatic mode, need to support the driver in resuming the driving task, by addressing problems like lack of attention, low situation awareness and skill reduction. According to the HATRIC project [11], there are three particular reasons for working with HMI for automation in relation to safety: (i) optimizing hand-over of control, (ii) minimizing negative effects of automation induced behaviour, and (iii) increasing usage by means of improved user experience. The fact is that HMI design strongly affects the driver's sense of safety: since perceived safety of the user highly correlates with his trust in technology [12], it is crucial to develop a framework for HMI assessment and to identify major factors affecting driver's trust in autonomous vehicles. From a human factor perspective, the design of automation systems so that drivers fully understand the capabilities and limitations of the vehicle and maintain situational awareness of what the vehicle is doing (and when manual intervention is needed) is currently a fundamental issue [13].
• Develop an ethical automated-decision-support framework, covering liability concerns and risk assessment: Trustonomy investigates liability concerns, compatible insurance models, ethical decision-making and auditability mechanisms when ambiguities arise. For instance, who is to blame when an RtI is not successfully completed, resulting in an accident? What is the (legally and ethically) suitable course of action in a situation whereby the ADS is about to reach its system limit, but driver state monitoring suggests that a driver intervention would also fail? Moreover, Trustonomy investigates precursors and forecasting models to issue early RtI warnings and provide more time to the human driver to intervene. In addition, it generates emergency trajectory possibilities in case of ambiguities (i.e. when an accident is impossible to avoid but multiple options exist). In order to achieve that, there is need for quantitative risk assessment of potential threats. Beyond the simple risk matrices traditionally used, with well-known defects [14], Trustonomy employs risk maps, following a paradigm already experimented in aviation, leading to improved safety results and costs [15], while a new approach of adversarial risk analysis [16] is used to encounter threats related to malicious attacks to the systems and algorithms regulating automated driving.
• Develop novel Driver Training Curricula for human drivers of ADS: Numerous EU-funded projects have aimed at designing real-life/simulation-based training modules targeting Advanced Driver Assistance Systems (ADAS) [17] and creating new training methodologies to cope with the rapid evolution of active safety systems [18], nevertheless, no Pan-European actions were made to fully acknowledge and include the training on handling these systems into the training curricula. This is especially important in the context of the OEM driving automation-targeted technological race, which is going to end up in deploying L3 vehicles already in 2018 [19] and building the capacity for L4 systems deployment in a 5-year horizon. If this pace is not slowed down by international legal restrictions, it will directly influence the driving behaviour of people, who were so far used to drive in the traditional, non-automated manner. The need for reinventing driver training in the ADS context has been repeatedly underlined by the EU experts [20,21]. Different research studies show that even the use of basic driving assistance systems like Adaptive Cruise Control (ACC) affect the driver's cognitive abilities and overall performance. Although it may reduce workload and stress while being on   the road, the situation awareness becomes negatively affected at the same time [22]. Drivers have a tendency to over-rely on the capabilities of such systems and negatively adapt to the new, less demanding conditions.
• Define a Driver Intervention Performance Assessment (DIPA) Framework: It is assumed that driver will become no more than a passenger during periods of automated driving mode performed by CAD vehicle. While driving needs constant monitoring, analysing and decision making to ensure safety [23], driver's role during autonomous driving will become out-of-the-loop, and he may not have enough information to maintain control on operational and tactic level of driving [24]. A driver may be under the influence of two main effects of disengagement: distraction and fatigue. Both may have negative impact on a driver, fatigue might i.e. diminish attention capabilities, and distraction might cause i.e. sharing attention between two tasks [25]. Furthermore, if the driver performs non-driving related task, the effects of its influence may last even up to 15s after its cessation [26]. This provides evidence that driver monitoring will be crucial in at least two phases of take-over: (i) pre-RtI phase when the driver will need to gain information of the current road situation, planned manoeuvres, etc.; (ii) take-over phase, when the driver will start to control the vehicle himself.
• Measure performance, trust and acceptance (simulations and field trials) of human drivers of ADS: There are a number of challenges associated with the concept of trust, with particular reference to the trust a driver has in an ADS. Unlike other "driver states" such as fatigue [27] or high workload [28], there exists no reliable physiological measure of trust. Lack of trust in ADS may induce anxiety in certain situations, resulting in increases in arousal that are able to be detected by physiological indicators, but still, it is difficult to correlate this observed arousal with reduced trust. The challenge is to design ADSs which are trusted appropriately: drivers have to trust them enough to glean all the promised benefits of, for example, traffic efficiency. Several studies have shown that trust is a key determinant for the adoption, intention to use and reliance on automated systems [29]. For sure, "operators tend to use automation that they trust while rejecting automation that they do not" [30]. On the other hand, over-reliance on automation is also not desirable and may lead to situations whereby drivers cognitively distance themselves so far from the driving task that they encounter difficulties in the transition periods. Such over-reliance was cited as one of the causes of the Tesla crash in 2016, with the NTSB noting "the operational design of the Tesla's vehicle automation permitted the car driver's overreliance on the automation, noting its design allowed prolonged disengagement from the driving task and enabled the driver to use it in ways inconsistent with manufacturer guidance and warnings" [31]. The trust that an operator has in a system is not binary; it can be situational as well as dynamic [32] and the challenge is to design an ADS that engenders trust at an appropriate level for any given situation.

III. APROACH
To address the challenges identified in the scope of intervention performance assessment, user trust and acceptance, Trustonomy adopts an integrated approach, where ADS-related state-of-the-art or emerging technologies and solutions are tested and evaluated with real users and nontechnical experts.
In the following paragraphs we further analyse the proposed approach, focusing on the objectives described in the previous section.

A. Driver State Monitoring
Trustonomy investigates the suitability and personalisation potential of various (combinations of) DSM techniques, by measuring and inferring: (i) sensory state, which affects the ability of the human subject to perceive the RtI and the surrounding contextual conditions; (ii) motoric state, in order to identify a body state that can be characterised as out-of-driving position; (iii) cognitive state, which affects the ability for applying attentional resources to perform the intervention; (iv) arousal level, which deteriorates when there is nothing to do for a long time; (v) emotional state, which is also considered explicitly, as it cannot be presupposed that rational behaviour lies at the heart of all decisions and actions.

B. HMI Design Factors
Trustonomy investigates the suitability and personalisation potential of various multimodal HMIs for maximising driver intervention performance, trust and acceptance, including: a) Visual factors (position and size of visual indicators, icons and colours, blinking); b) Auditory factors (loudness, tonal pattern, voice); c) Haptic factors (bodily part, i.e. hand, foot, thigh, vibration pattern, mid-air HMI feedback); d) Timing of onset of RtI; e) Content of HMIs, ranging from automation mode change (e.g. temporal function halt, malfunction), RtI message types (e.g. "please take over!"), intervention action indications (e.g. "hands on wheel!"), to HMIs to display system state and HMIs to indicate system reliability, etc.

C. Risk Assessment
Trustonomy aims at identifying first a detailed catalogue of threats that might affect automated/semi-automated driving and undermine public trust and confidence in this transportation means. Based on such catalogue, it shall undertake a risk matrix approach to screen the most worrisome threats and then perform a detailed quantitative analysis over such list, producing a risk mapping with a full quantitative risk assessment model. Adversarial risk analysis models will be developed to support automated driving, helping in better forecasting how other road users behave, and underpinning improved automated decision-making in driving. Finally, the robustness of algorithms supporting automated driving towards attacks will be explored; as an example, an artificial vision algorithm could be hacked and a STOP sign of a road could be misinterpreted leading to chaotic situations. This will lead to the assessment of such algorithms from an adversarial machine-learning perspective.

D. Early Warning
Based on the risk assessment described above, Trustonomy will define and study precursors of such threats and build forecasting models to issue the RtI warnings as early as possible, in order to provide more time to the human driver to intervene. Essentially, several relevant signals will be tracked, monitored and forecasted based on dynamic models against several thresholds leading to RtIs. Such forecasts will be issued several instants ahead in such manner that if the threshold is expected to be reached by the prediction intervals, the RtIs would be issued.

E. Trajectory Planning
In the case of emergency trajectory planning, the generation of trajectories will be done by comparing different planning algorithms such as parametric planning or graph search planning, with the objective of mitigating the accident consequences. The planning method will be multi-objective, to generate a set of optimal trajectories according to cost functions depending on the accident consequences (fatalities, social cost, financial cost, etc.); genetic algorithms will be used to determine these planned trajectories. A panel composed of experts and regular drivers, cyclists and other road users will be asked to select which is the best trajectory from the ones proposed by the algorithm; this ethical question will then be partially solved by such democratic vote.

F. Driver Training (curricula, methods, material)
Trustonomy identifies the need to prepare newly trained drivers for higher (L3-L4) stages of driver automation in which efficient driver-vehicle interaction will be the key to increasing road safety. To this end, a thorough road-safety targeted risk mapping with respect to both ADS performance as well as driver reception and psycho-motoric performance will be made. This will allow to identify specific priorities to be covered in the course of the training. For each of the identified problems, an individual training method will be developed, and applicable ICT-based training tools will be selected, so that a full training curriculum for human drivers of ADS is composed and tested through real-life piloting (involving passenger vehicles, light/heavy freight, public transport, etc.).

G. Driver Intervention Performance Assessment
DIPA involves the definition of relevant objective measures to assess the quality of intervention performance, such as driver take-over time from onset of RtI, driver intervention time, control stabilisation time, remaining action time, as well as subjective measures for the quality of intervention performance. It consists of a set of measures to determine whether the driver is able to perform an intervention in a safe way or it is worth maintaining the control in ADS and perform a Minimal Risk Manoeuvre.

H. Driver Trust
One of the most important goals of Trustonomy is to assure that automated vehicles are being trust by drivers. A suite of driving simulator studies will be carried out to investigate how a range of users of automated vehicles learn to trust the key features, the situations in which that trust diminishes and how degraded levels of trust can be boosted in an accelerated manner. A toolbox of driving scenarios will be developed which can be used to measure, maintain and, where necessary, increase levels of trust to the point where maximum benefits of automation can be accrued, without the driver becoming over-reliant. Potential research questions include: what aspects of automation (sub-functions) are more susceptible to loss of trust? Which functions can "degrade gracefully" without substantial loss of trust? Can the ADS be "programmed" to self-evaluate its reliability and thus predict the real-time trust that an operator has in it? What interventions could help an operator regain trust?

IV. TRUSTONOMY OVERALL ARCHITECTURE
In order to perform the assessment of the emerging technologies presented in the previous section, we introduce the conceptual architecture depicted in Fig. 2. As depicted, the approach followed for the definition of the conceptual architecture was a mixture of top-down and bottom-up approaches. The processes/actions used for the validation of the conceptual architecture were derived from the user requirements, which were in turn used for the definition of the use cases (top-down). Concurrently, the functionality of the different components was initially defined using the findings from the state-of-the-art review and was adjusted to meet any requirements that were not originally taken into consideration (bottom-up).
As it has been previously highlighted, the project produces outcomes in different ADS-related design domains. Fig. 2 illustrates a conceptual design of the Trustonomy architecture. The upper part of the figure depicts the Trustonomy Frameworks. The different frameworks, namely DSM Assessment Framework, HMI Design Assessment Framework, Automated Decision Support Framework, Driver Training Framework, Driver Intervention Performance Assessment Framework and Trust and Acceptance Measurement Framework are the main outcomes. These frameworks lead to stand-alone tools that can be used for the assessment and evaluation of ADS specific parameters related to performance, risk and trust assessment. Obviously, the domain of analysis of each framework is different and, for this reason, the resulting tools are independent and can be used as stand-alone solutions. To perform the analysis and assessment, the Trustonomy frameworks are based on data that are collected real-time on the Trustonomy pilot sites or on datasets that have been prerecorded during specific scenarios of interest. The Trustonomy pilot sites involve different conditions (e.g. road conditions) and different vehicles (public transport buses, passenger vehicles, freight transport cars and driving simulators). Additionally, multiple configurations and technologies (sensors, DSMs, HMIs) are deployed within the different vehicles to allow the monitoring, study and evaluation of the vehicle state and the driver condition and behaviour.
To support the management of the multiple data streams collected from the pilots and streamed to the data analytics processes and applications, Trustonomy is based on a data management layer that acts as the middleware between the multiple data sources of the project and the Trustonomy Frameworks. The same set of tools will be used for the management of various pre-recorded datasets that will be used for the analysis performed by the Trustonomy Frameworks. The above lead the development of the initial specifications for the individual Trustonomy tools with primary aim to ensure the availability of data sources that each component requires in order to function. As part of this activity, input and output data sources for each component were identified and an overall initial conceptual architecture was drawn up.
Finally, as illustrated on the right part of Fig. 2, a Trials Support Tool will be developed, aiming to assist the execution of the Trustonomy trials. Fig. 3 presents the functional architecture of Trustonomy, with the individual Trustonomy frameworks and their internal functions. The Trustonomy architecture consists of the following frameworks:

A. Functional Architecture
• DSM Assessment Framework: Assess the performance of one or more DSMs.
• HMI Design Assessment Framework: Assess the performance of different HMI designs.
• Automated Decision Support Framework: Undertake the decision of issuing a Request to Intervene or preserve an autonomous driving mode and, if so, plan the driving decisions (e. g., trajectory) accordingly.
• Driver Training Framework: Assess and validate the driving training curricula.
• Driver Intervention Performance Assessment Framework: Assess the driver's ability to intervene in case this is needed.
• Trust and Acceptance Measurement Framework: Produce methodologies to assess trust and acceptance in ADS.
The Data Management layer is not amongst the main outcomes of the project, but it is a layer encompassing functions related to data management procedures, acting as an enabling technology for the Trustonomy Frameworks.
V. CONCLUSIONS This paper elaborated upon the Trustonomy vision on maximising the safety, trust and acceptance of automated vehicles. The key benefit of the proposed approach is that it encounters all the challenges that arise in the dynamic drivervehicle interaction scheme that we see in today's mixed traffic environments. Specifically, an emphasis is given on the driver state monitoring systems, the application of human-machine interfaces, the use of risk assessment for tracing potential threats, the necessity of reinventing driver training material for autonomous vehicles, the measuring of driver's intervention performance, and finally the necessity to measure performance, trust and acceptance. The conceptual and the functional architecture of the envisioned system have been presented, while further research activities include the implementation of the individual Trustonomy frameworks and then the testing and validation of the discussed approach in extended pilots in fully operational environments, evaluating the performance and impact of the proposed approach. These activities will be carried out through the duration of the Trustonomy project.