On Highly-Skilled Autonomous Competition Vehicles: An FSM for Autonomous Rallycross

This paper introduces a novel Finite State Machine (FSM) which behaviour resembles professional Rallycross drivers. Essentially, the proposed FSM integrates different driving states like racing-line-based and autonomous-drifting to drive the car at the limits of handling through tight track layouts and changing road surfaces, as professional drivers do. In particular, the drift stabilisation and agile manoeuvring tasks are realised by coordinating the intervention of four independent Electric Motors (EMs), Steer-by-Wire (SBW), and Electro-Hydraulic Braking (EHB) systems, while the friction adaptation attribute is achieved by means of a machine-learning-based approach. The ability of the FSM to drive at the limits of handling through a high-demanding virtual Rallycross track is assessed in IPG-CarMaker. The results evidence the potential of the proposed FSM for the development of future Highly-Skilled Autonomous Vehicles (HSAV).


I. INTRODUCTION
During the last years, the behaviour exhibited by professional competition drivers has been a subject of study for a relevant number of researchers [1], [2], [3], [4]. Overall, it is believed that a more comprehensive understanding of racing techniques like drift control or agile manoeuvring could help to develop enhanced chassis systems, and eventually, contribute to the creation of a new generation of Autonomous Vehicles (AVs). Such vehicles should be able to exhibit a robust fulllevel autonomy [5], control the car at the limits of handling, perform in drastically-different terrains, and adapt to timevarying friction conditions. In addition to the recent academic interest on the topic, the realisation of the previous goal has lead to the birth of the "autonomous motorsport" concept. On this basis, organisations like Roborace are currently working towards the development of sophisticated platforms to test AV technology on highly-adverse and demanding competition environments [6].
Among the wide range of current motorsport disciplines, professional Rallycross stands out from the rest due to its toughness, Figure 1. In particular, drivers compete along a Rally-like closed track, adapting their behaviour during abrupt friction transitions (e.g. from tarmac to gravel) and stabilising the vehicle at high body slip angles along tight track segments. From an automotive engineering perspective, Rallycross unites two ingredients that are of particular interest for chassis system development: high body slip stability and adaptation to drastically-different terrains and time-varying friction conditions. Unfortunately, the current literature lacks dedicated works where state-of-the-art AV technology is subjected to these high-demanding motorsport scenarios. This work has been prepared with the aim to shed some light on the elaboration of future autonomous motorsport platforms for full-level AV technology development. The major contribution of this work lies in: (a) for the first time, a Finite State Machine (FSM) is proposed to drive a Multi-Actuated Ground Vehicle (MAGV) along a complex virtual Rallycross track, combining racing-line-based and drifting-like driving modes and (b) a novel machine-learning-based road friction recognition algorithm is integrated into the FSM to realise the friction adaptation feature. The rest of the paper is organised as follows.
In Section II, the proposed FSM is presented and the different control layers and path-following algorithms are explained. The analysis of the FSM continues in Section III, where the proposed friction adaptation schemes are detailed. After that, the simulation results obtained in IPG-CarMaker are discussed in Section IV. Finally, conclusions and future research steps are provided in Section V.

II. FINITE STATE MACHINE
The FSM introduced in this paper is depicted schematically in Figure 2. Overall, two modules can be distinguished in this structure: a layered control block and a friction learning module. The layers of the former block are detailed in the following, while the friction learning module is presented in Section III.

A. Driver Selector layer
In the first layer, a suitable driving mode is selected depending on the road curvature information. Specifically, for tight road segments in which agile manoeuvring and high body slip control are advantageous for minimum-time cornering [7], [8], the high-body-slip driver is selected. For higher radii (e.g. straight-line driving), the racing-line-based driver is chosen. The road geometry information is supplied to this layer as a vector κ = {κ di } that contains the discrete values of the reference trajectory curvature (κ) at a distance (d i = 0, 10, 20, 30 metres) ahead of the vehicle. This information is assumed to be supplied by an upper-level perception layer that may use machine vision, LiDAR, RADAR sensors or a combination of all for this purpose.

B. Path-following layer
Once a suitable driver model has been selected, the highlevel control references are generated to keep the vehicle along the reference path. Following a methodology similar to [9], two path-following approaches are adopted: racing-line-based and drifting-based reference generation.
1) Racing line: The racing line path-following computes the steering commands δ adopting a simple lateral-deviationbased feedback control law of the form, where e ψ and e y are the yaw and lateral deviation errors of the vehicle with respect to the reference line, and K ψ ,K ey are suitable feedback gains. With respect to the longitudinal control action, a racing like switching (ACCELERATION/BRAKING) strategy is adopted for simplicity. Essentially, when a turn is exited and the racing-line-based driver is selected, maximum torque is requested to the EMs to achieve maximum acceleration. In parallel, a minimum braking distance metric (d min ) is computed to determine the next braking point. A constantdeceleration model of the form, is adopted for this purpose. In this case, v x is the vehicle velocity, v ref,i is the reference velocity at the i − th corner entry, and a x,dec is the maximum vehicle deceleration. At each time step, the distance with respect to the turn entry of the i − th corner (d target ) is compared to the previous metric (d min ). When the condition (d target < d min ) is fulfilled, the braking action is triggered, and the braking torque demand is computed using a proportional feedback law, with K brk being the braking feedback gain.
2) High body slip: The path-following-drifting concept has been introduced in previous works [10]. Essentially, this layer corrects the open-loop reference path curvature and provides a corrected valueκ to the low-level drift control system with the aim to produce a trajectory that matches the reference path. The proposed path-following approach relies on the information supplied by a set of preview-distance points (P i ), Figure 3. Fig. 3. Scheme of the proposed drift-path-following control law.
In this manner, the corrected curvature κ is computed adopting a feedback control law (κ = κ − ∆κ), with ∆κ being proportional, derivative and integral gains.

C. Vehicle Dynamics (VD) control layer
This layer contains the control algorithms necessary to fulfil the requests dictated by the path-following layer.
1) Continuous Wheel Slip (CWS) control: To start with, the Traction Control System (TCS) and Anti-Lock Braking System (ABS) functions are incorporated into the FSM with the aim to maintain optimum longitudinal slip values during emergency braking and harsh acceleration conditions. Following the methodology introduced in [11], these functions are realised by means of Integral Sliding Mode Control (ISMC).

2) Autonomous Drift Control (ADC):
The ADC function is implemented by means of a centralised Linear Quadratic Regulator (LQR) controller. Other control techniques (e.g. Model Predictive Control [10]) have been also explored by the authors. Despite MPC can be advantageous in the presence of actuator constraints, LQR is adopted in this work with the aim to maintain a minimum computational load. Essentially, the LQR computes the individual torque (T i ) and steering inputs (δ) to track a set of vehicle dynamics references (x) that depend on the reference body slip angle (β ref ) and the corrected curvature (κ) dictated by the High-Level Drift Reference layer (i.e. x = f (β ref , κ)). Additional details can be found in [10], where a similar control structure is adopted.

D. MAGV layer
Finally, the individual torque (T i ) and steering (δ) requests demanded by the VD control layer are realised by the Multi-Actuated Ground Vehicle (MAGV) actuators. In this layer, amplitude and slew rate constraints are considered for the SBW system. With regards to the EMs, the maximum available torque is extracted from torque-versus-speed curves. Additional details regarding the MAGV constraints considered on the virtual vehicle model are omitted in this paper due to confidentiality reasons.

III. FRICTION LABELLING, LEARNING, AND ADAPTATION
As can be seen in Figure 2, the proposed FSM incorporates a friction learning module to enhance the system robustness to time-varying friction conditions. A two-step approach is proposed in this work in order to (1) label and learn an unknown race track and (2) handle time-varying friction conditions during the race.

A. Friction Labelling (Warm-up lap)
Initially, no prior knowledge regarding the race track friction conditions is assumed. As Rallycross tracks are composed of more than one surface (tarmac/gravel), an initial clustering and learning process of each road terrain is necessary to derive the initial control references. The proposed friction clustering process is depicted schematically in Figure 4. Specifically, during the warm-up process, the vehicle acquires a cloud of friction versus slip values from several braking and steering inputs performed along the unknown track (1)(2). After that, a Self-Organising Map (SOM) is employed to distribute a set of weights along the logged cloud of points (3). The former weights are latter classified adopting a k-means clustering approach (4). In this case, the labels considered are tarmac and gravel. In particular, dry tarmac is characterised for having a grip potential factor µ max ≈ 1 and low optimum tyre slip ranges (e.g. α opt ≈ 5 degrees) while dry gravel presents a lower maximum grip factor µ max ≈ 0.8 and significantly higher optimum slip values (e.g. α opt ≈ 50 degrees), see Figure 7. This two-step clustering approach was adopted following recommendations from [12] with the aim to reduce the negative influence of outliers points. The logged data is labelled according to the clustered SOM weights and an MF tyre model is fitted with the data from each surface (5). Finally, the initial FSM references are derived from the friction models extracted from this process.
From an aerospace or military perspective, this step may be seen as an "exploration" mode in which the system labels and learns the terrain of a certain route for the first time.

B. Friction Adaptation (Minimum-time route execution)
The race starts once the terrain has been labelled and the Highly-Skilled Autonomous Vehicle (HSAV) has been initialised. For simplicity, it is considered that the HSAV is able to locate each of the road segments previously labelled (gravel/tarmac) along the track. This may be achieved by adopting a GPS-based location approach or a braking-based road identification routine [13]. As the friction conditions may change considerably during the course of the race (e.g. due to rain or tyre degradation) it is necessary to adapt the vehicle references to the time-varying friction conditions. Two adaptation strategies are presented in this work depending on the type of surface considered on the track (rigid surface or loose surface).
1) Rigid surface adaptation: In this case, the magic formula grip scaling approach is adopted [14], and the friction variations are monitored by means of a friction estimate (μ est ). This estimate is computed during each ABS intervention aŝ µ est = a x /g, with g being the gravity constant, and during sustained drifting manoeuvring as, (5) where A yc is the vehicle centripetal acceleration, and A yc,min , A yc,max are the centripetal acceleration values expected from two extreme friction conditions (e.g. µ max = 1 µ min = 0.7). The raw estimateμ est is filtered adopting Recursive Least Squares (RLS) and used to adjust the maximum vehicle deceleration (a x,dec ) and the tarmac drifting references of the FSM using a linear interpolation approach. The tarmac drifting references corresponding to different friction levels as well as the associated centripetal acceleration values were computed offline from the tarmac friction model obtained in the warm-up lap.

2) Loose surface adaptation:
In what concerns surfaces like gravel or deep snow, a single grip scaling factor is not sufficient to characterise the wide range of shapes that the friction curve can adopt depending on factors like the softsoil terrain content. In these conditions, authors propose the machine learning approach depicted in Figure 5. Before the start of the race, a set of feedforward friction NNs are trained with synthetic data obtained from the friction models fitted during the warm-up lap. These friction NNs are employed to generate the initial gravel drifting references. Once the race is started, each time the HSAV covers the loose surface segment, the weights of the NN friction structures are updated with new friction data using the Levenberg-Marquardt (LM) backpropagation training algorithm. Specifically, four NN structures are employed to model the tyre friction in the longitudinal and lateral combined-slip cases (i.e. µ x = F x /F z and µ y = F y /F z , with F x , F y , F z being the tyre forces in the three axes). In the next step, a synthesised vehicle planar dynamics model is updated with the new friction characteristics learned by the NNs and a Sequential Quadratic Programming (SQP) optimisation routine is launched to find the drifting references that maximise the vehicle lateral dynamics [10]. Finally, it is important to remark that the previous steps (NN training and SQP optimisation) need to be executed within the time interval required to cover the tarmac segment (t 1 − t 0 ), Fig. 5

-bottom.
In terms of aerospace or military applications, this step may be seen as a minimum-time route execution problem in which the HSAV seeks to cover a certain predefined route in the minimum possible time. In such a scenario, the target route may be continuously passed by several vehicles with the ability to log and send data to a central base station using vehicle-to-infrastructure (V2I) communication technologies. Such information might be employed to monitor the friction characteristics of the path and generate updated system references to be loaded into new incoming vehicles.

IV. RESULTS: VIRTUAL RALLYCROSS TRACK
The proposed system was tested in a virtual Rallycross track modelled in the vehicle dynamics simulation software IPG-CarMaker, Fig. 6. For the sake of simplicity, a unique track layout was considered, and alternative paths for "joker laps" [15] were not modelled. The FSM explained in the previous sections was implemented in Stateflow/Simulink and a sportsclass virtual MAGV was modelled in CarMaker. The chassis model validity was verified by means of experimental steadystate and step steer manoeuvres. Additional details regarding the virtual MAGV are omitted here due to confidentiality reasons. Regarding the FSM feedback, the vehicle dynamics signals and tyre forces were directly taken from the Car-Maker/Simulink framework. Virtual sensing solutions to obtain these from inexpensive vehicle dynamics measurements are currently being explored and will be implemented in future steps. Finally, the road geometry information was accessed from the CarMaker virtual track by means of RoadProperty sensors and the relative distance between the vehicle position and the braking beacons located at each turn entry was obtained from a DriverAssistance sensor [16].

A. Race scenario: It rains!
In order to study a high-demanding scenario, the timevarying friction models illustrated in Figure 7 were implemented in the virtual track. In brief, at the start of the race (t = 0) dry conditions are assumed on both surfaces (red friction curves). The tarmac surface is modelled adopting an isotropic MF tyre model and a maximum grip value µ max = 1 [8]. The MF parameters of the gravel terrain at t = 0 were fitted from constantspeed steering-ramp tests carried out on a dry packed gravel platform. After 10 minutes of rain (t = 600s, cyan friction curves), the tarmac surface is slippery (µ max = 0.7) [8], and the dry packed gravel has turned into an extreme off-road terrain with a reduced friction stiffness and a high soft-soil content (large bulldozing effect) [8]. The change between this conditions is assumed to be progressive, and modelled in IPG-CarMaker by means of the linear-interpolation friction surfaces depicted in Figure 7. Finally, a rough road profile was added to the gravel terrain adopting the approach presented in [10], [13] in order to model realistically severe vertical disturbances.
1) Warm-up lap: The virtual vehicle was manually driven along the virtual track during the warm-up lap. Several braking and steering inputs were performed in order to acquire sufficient friction information. This process was carried out with a static driving simulator setup [13]. This step may be performed manually on current driverless motorsport vehicles [6] adopting a virtual reality driving platform. Apart from this, a methodology to perform the warm-up lap in autonomous mode is currently under investigation. After that, the tarmac drifting references were derived for different grip scaling factors (from µ max = 1 to µ max = 0.7) and implemented in n-D look-up tables. Regarding the initial gravel drifting references, these were derived from 2-8-1 friction NNs trained with synthetic data from the friction models obtained in the warm-up lap. This number of hidden neurons was set in order to minimise overfitting issues and was found to be suitable for the friction characteristics considered in this problem.
Finally, different longitudinal slip references λ ref were set for the TCS and ABS systems, with λ ref being defined according to [8]. Specifically, in tarmac conditions λ ref was set to 10% to maximise the longitudinal friction and retain some vehicle manoeuvrability. On the other hand, during offroad driving, λ ref was increased significantly (up to 80%) with the aim to take advantage of the bulldozing braking force. A more sophisticated combined action of the steering, braking and acceleration modes of the racing-line-based driver will be explored in the future in order to further minimise the turnentry and turn-exit time.
2) Race (Friction adaptation enabled): After initialisation, a race consisting of six laps was simulated. The trajectories of the FSM with the friction adaptation algorithms enabled are illustrated in Figure 8. As can be noticed, the vehicle remains within the track limits during the full race (6 laps t f inal = 701.4s). The Root Mean Square Error (RMS) and Maximum Absolute Error of the vehicle lateral deviation with respect to the road reference path (centre line) for each lap are given in Table I. In all the laps, the maximum deviation is maintained below the maximum semi-track width (7.5m) in spite of the time-varying friction conditions.
The trajectory of the third lap (chosen randomly for illustration purposes) is depicted in Figure 9. During the tarmac segment, the HSAV computes continuously the friction   Fig. 10-top) and adapts the tarmac friction references accordingly. In parallel, the friction adaptation loop depicted in Figure 5 is executed (Fig. 10-bottom). Overall, an acceptable average time update of 30s (within the tarmac time window (t 1 −t 0 ) ≈ 40s, Fig. 5) was obtained for the complete NN weight update and SQP optimisation using a computer equipped with an Intel i7 8th generation processor and 8GB RAM memory.
3) Race (Friction adaptation disabled): Finally, in order to evidence the influence of the proposed friction adaptation strategy, the same race simulation was repeated with the adaptation function disabled, maintaining the system references derived from the friction models obtained in the warm-up lap. Expectedly, the vehicle was unable to cover the full race and left the road during the third lap, Figure 11. The lateral deviation error metrics increased progressively (Table I) until a maximum absolute error of 13.28 metres occurred in the third lap and the simulation was stopped. These results illustrate the difficulty of maintaining a vehicle drifting at the limits of handling during time-varying friction conditions along tight road geometries.

V. CONCLUSIONS
In this paper, a novel FSM to achieve highly-skilled autonomous driving has been proposed. Moreover, a novel friction adaptation strategy based on feedforward Neural Networks has been incorporated into the FSM in order to achieve the system adaptation to time-varying friction conditions. The FSM has been simulated in a virtual Rallycross track modelled in IPG-CarMaker formed by time-varying tarmac and gravel friction models. The results evidence the adaptation ability of the proposed FSM, which is able to maintain the vehicle on the track at the limits of handling in spite of the changing friction conditions like professional drivers do. Overall, the authors believe that the methodology described in this paper to design adaptive autonomous vehicles can significantly contribute to the development of future Highly-Skilled Autonomous Vehicles. Finally, tyre force virtual sensing solutions will be explored in future steps of this research with the aim to achieve the proposed adaptation skills from a set of inexpensive onboard measurements.

ACKNOWLEDGEMENT
This project is part of the Interdisciplinary Training Network in Multi-Actuated Ground Vehicles (ITEAM) European program and has received funding from the European Union Horizon 2020 research and innovation program under the Marie Sklodowska-Curie grant agreement No 675999.