MOCA-MAN: A MObile and reconfigurable Collaborative Robot Assistant for conjoined huMAN-robot actions

The objective of this paper is to create a new collaborative robotic system that subsumes the advantages of mobile manipulators and supernumerary limbs. By exploiting the reconfiguration potential of a MObile Collaborative robot Assistant (MOCA), we create a collaborative robot that can function autonomously, in close proximity to humans, or be physically coupled to the human counterpart as a supernumerary body (MOCA-MAN). Through an admittance interface and a hand gesture recognition system, the controller can give higher priority to the mobile base (e.g., for long distance co-carrying tasks) or the arm movements (e.g., for manipulating tools), when performing conjoined actions. The resulting system has a high potential not only to reduce waste associated with the equipment waiting and setup times, but also to mitigate the human effort when performing heavy or prolonged manipulation tasks. The performance of the proposed system, i.e., MOCA-MAN, is evaluated by multiple subjects in two different use-case scenarios, which require large mobility or close-proximity manipulation.


I. INTRODUCTION
The introduction of collaborative robotic technologies embedded with rich actuation and sensing systems opened a new horizon of automation opportunities for agile manufacturing, healthcare, and even disaster response. This was due to their main distinctive features that promoted safety when working with their human counterparts [1], and enabled adaptation to their unknown surroundings [2], [3].
Cobots, on the other hand, have demonstrated a high potential for improving productivity [10] and contributing to better ergonomics and comfort of their human counterparts [11]. Cobots can have fixed or mobile bases, and function autonomously or in collaboration with humans to handle manipulation tasks. The latter can cover a range from coexisting agents [12], [13], when humans and robots share a common workspace with no direct interaction, to direct collaboration [14]- [16], when human and robots come to a contact to perform a common task [1]. This   For the time being, the functional capacities of the supernumerary limbs (or exoskeletons) and cobots are perceived as complementary and synergistic, especially in industrial domains. However, this consideration adds costs and requires a range of personnel expertise required for training and maintenance of the substantially different systems. An effective solution to these shortcomings, through lean thinking approach [17], would be a unified system that can reconfigure quickly between cobots and supernumerary limbs, to be able to cover a large range of industrial tasks' requirements.
Accordingly, this paper aims to exploit the reconfiguration potential of a MObile Collaborative robot Assistant (MOCA), to subsume the advantages of cobots and supernumerary limbs. This impels that, we aim to create a system that can function autonomously or in close proximity to humans, or be physically coupled to the human counterpart as a supernumerary body (MOCA-MAN), when necessary. The physical coupling is achieved through a mechanical admittance interface that can enable MOCA whole-body movements as desired: through a hand gesture recognition system, the controller can give higher priority to the mobile base (e.g., for long distance co-carrying tasks) or the arm movements (e.g., for manipulating tasks), when performing conjoined actions. Meanwhile, the admittance interface translates the user forces to the end-effector trajectories in space to enable the execution of joint actions.
As a result, the user does not have to carry the weight of the supernumerary limb, and can come to connect with it only when necessary. Otherwise, MOCA can reconfigure to a mobile collaborative robot and perform its routine work. This consideration not only can contribute to the user's comfort and increasing productivity, but also to reduce equipment waiting and setup times, typical of small and medium enterprises [18]. We evaluated the performance of the proposed MOCA-MAN in two different use-cases, which required large mobility or close-proximity manipulation. The results reveal the high potential of the proposed reconfigurable interface in manufacturing environments.

A. System Hardware Integration
The core component of the robot hardware in this paper is the new versatile cobot platform MOCA as shown in Fig. 2 that we recently presented in [19]. We additionally developed a new admittance interface, which is positioned between the robotic arm and the SoftHand. The details of the admittance interface will be presented in Section III-A.
A whole-body impedance controller is developed to deal with the different causalities of the admittance controlled Robotnik SUMMIT-XL STEEL (3-Degrees of Freedom (DoFs)), the torque controlled robotic arm system (7-DoFs). Meanwhile, the desired behaviour of MOCA-human conjoined action is achieved by the admittance interface. The developed whole-body impedance controller can achieve the desired impedance behaviour at the end-effector as well as the redundant DoF. This is a crucial requirement not only for ensuring human safety, but also for executing complex manipulation tasks autonomously or in conjoined humanrobot situations.

B. Whole-body Impedance Controller
Let us consider the mobile-manipulator with 3-DoFs (rigid body motion) at the mobile base and n-DoFs at the manipulator. We define the generalized coordinates q = [q T v q T r ] T ∈ R 3+n , with q v and q r the coordinates of the mobile base and the manipulator. The dynamics equation of the combined mobile-manipulator, by assuming an admittance causality for the mobile base (which is velocity controlled), can be described by where M adm ∈ R 3×3 and D adm ∈ R 3×3 are the virtual inertial and virtual damping terms for the admittance control of mobile base,q v ∈ R 3 is the velocity of the generalised motion of mobile platform, Γ Γ Γ vir v ∈ R 3 and Γ Γ Γ ext v ∈ R 3 are the virtual and external torques. On the other hand, M r ∈ R n×n is the symmetric and positive definite inertial matrix, C r ∈ R n×n is the Coriolis and centrifugal matrices, g r ∈ R n is the gravity vector, Γ Γ Γ r ∈ R n and Γ Γ Γ ext r ∈ R n are the joint torque vector and external torque vector of the robotic manipulator, respectively.
By letting x ∈ R 6 be the task coordinates in Cartesian space, the desired task-space dynamics behaviour in response to the external wrench F ext ∈ R 6 , (leading to the external torques (1)), can be obtained by wherex = x − x d is the Cartesian error from the desired task x d , and K ∈ R 6×6 and D ∈ R 6×6 are the desired Cartesian stiffness and damping matrices, respectively. Λ Λ Λ(q) ∈ R 6×6 is the Cartesian inertial and µ µ µ(q) ∈ R 6×6 the Cartesian Coriolis and centrifugal matrices, respectively (see our previous work in [19] for more details). One crucial requirement for enabling intuitive human-MOCA conjoined actions is to be able to selectively assign larger mobility to the arm or the mobile base when a desired trajectory is executed at the end-effector. For instance, it is desirable to have a priority on the arm movements when performing a joint drilling task in a reachable workspace of the MOCA or human. This is to avoid unnecessary mobile base movements that can cause collisions with the human partner or the environment. On the contrary, when co-carrying objects in free spaces, a higher priority on the mobile base is preferable to make the coordination of the conjoined action more intuitive.
To achieve such behaviours, we implemented a weighted dynamically-consistent pseudo-inverse to apply the desired motion constraints through real-time variable weighting factors. The weighted dynamically consistent pseudo-inverse is defined asJ where is the weighted Cartesian inertia, J ∈ R 6×(3+n) represents the whole-body Jacobian matrix, M ∈ R (3+n)×(3+n) denotes the whole-body inertial matrix, and W ∈ R (3+n)×(3+n) is the diagonal and positive-definite weight matrix defined by W = diag [w 1 w 2 · · · w n ] , with w i ≥ 0. Consequently, a higher value of w i at the i-th joint will impede the motion of that joint, and W = I 3+n will make no effect on the motion mapping.
The whole-body Cartesian impedance controller's commanded torque for the main task can be computed using where x d ∈ R 6 is a vector of desired Cartesian position and orientation. x d in conjoined movements is obtained from the developed admittance interface, which will be explained in the following section.
To ensure human safety when performing conjoined actions, we implemented a soft null-space behaviour so that possible collisions will be responded with small interaction forces. This is achieved by creating a desired joint stiffness K n ∈ R (3+n)×(3+n) and damping D n ∈ R (3+n)×(3+n) behaviour where q d,0 ∈ R 3+n is the virtual equilibrium position. To avoid any potential conflicts with the primary task at the end-effector, the compliant joint behaviour torque (Γ Γ Γ n ) is projected using a dynamically consistent null space matrix N n [20] and provided to the whole-body impedance controller

III. HUMAN-MOCA ADMITTANCE INTERFACE
A. Hardware   Fig. 3 illustrates the developed human-MOCA admittance interface. The interface is designed as an extension link from the MOCA SoftHand flange to the human wrist. This mechanical connection is achieved by a magnetic clamp that enables to easy and fast attachment and detachment when needed, and enables to transmit the interaction forces applied by the human for the execution of conjoined human-robot movements. The magnetic clamp is considered not only for the fast reconfigurability, but also to ensure human safety when an unexpected MOCA behaviour is observed.
The admittance interface consists of three parts: the robot mounting part (A in Fig.3b), the wearable part (C in Fig.3b), and the magnetic clamp part (B in Fig.3b). The dimension of device is 271 × 140 × 102 mm and the weight is 0.47 kg which includes the FT sensor and the arm brace.
The robot mounting part is attached to the MOCA endeffector flange, and is placed between the PISA/IIT Softhand and Franka Emika Panda's wrist flange. The flange is designed in a way not to constrain robot end-effector movements when MOCA operates unaccompanied. This part is composed of a six-axis ATI Mini-45 Force-Torque (FT) sensor and a push switch. The FT sensor measures the human-robot interaction forces, when the human-intended forces are occurred in rigid structure during the execution of conjoined motions. Thanks to the FT sensor, it is able to distinguish them from the interaction forces that arise from interacting with the external environment, therefore, the human-intended forces are then transmitted through the admittance interface to obtain the desired trajectories for the conjoined motion by the admittance control algorithm (details in Section III-B). The FT sensor is displaced 210 mm from Σ EE along the negative y-axis. The push switch is placed in the last layer of the mounting part, to detect whether the wearable part is clamped or not through the magnetic clamp. This is also to enable fast switching between the autonomous and conjoined human-MOCA modes, and also to ensure human safety when the wearable part is released from the robot mounting part. Although the conjoined mode is terminated as soon as a user detaches his/her arm from MOCA, a time interval for 3 seconds is considered to make a safe action for him/her to move away from it.
The wearable part is assembled with a comfortable arm brace that a user can wear on the hand. To ensure that the interaction forces are directly transmitted to the FT sensor, the wearable part and human interface are connected tightly through three Velcro loop bands. The shape of the bottom surface is designed to adapt to the human body arm and wrist with a curved surface. Additionally, to enable intuitive control of the Pisa/IIT Softhand, the attachment parts are designed to align the human hand and the robot hand during the conjoined task executions.
The clamping force can vary based on the magnetic field's strength, that is decided due to the threshold of safe interaction forces. The magnetic interface includes three disk magnets of 12 mm diameter with 10 mm height. When two unequal magnetic poles attract each other at a distance of 4 mm, the attractive force is approximately 12.18 N. Accordingly, interaction forces can be transferred up to 36.54 N via the interface. Hence, when the human-inserted interaction

WHOLE-BODY IMPEDANCE CONTROLLER
x a ,ẋ a < l a t e x i t s h a 1 _ b a s e 6 4 = " a 0 X 7   p y U s D i R m K H 5 R 0 F C D R k Z e T a G T z h G k s 4 U g Y g T t a u B J p B D J F W C N R W C t X j y M u m 3 W 9 Z p q 3 1 z V u 9 c l n F U w R E 4 B i f A A u e g A 6 5 B F / Q A A g / g C b y A V + 1 R e 9 b e t P d 5 a U U r e w 7 B H 2 g f 3 w 6 R n W I = < / l a t e x i t >x ,ẋ force overcomes this threshold, the magnetic clamps is released and MOCA control is stopped for safety.

B. Admittance Behaviour for Conjoined Motion
The human-MOCA admittance interface is developed to enable conjoined movements when performing manipulation tasks or carrying loads. For this reason, we exploit an admittance control loop to transfer contact wrenches to desired end-effector movements, through an admittance model [21], [22]. The whole-body behaviour of the MOCA robot when following the trajectories can be regulated based on the task requirements and the mobility considerations, as explained in Section II-B.
Let F m = [f T m µ µ µ T m ] T ∈ R 6 denote the contact wrenches measured from the external FT sensor (mounted at the admittance interface), and ϑ ϑ ϑ d = υ υ υ T d ω ω ω T d T ∈ R 6 , the spatial desired velocity vector, where υ υ υ d and ω ω ω d correspond to linear and angular velocities, respectively. The dynamic relationship of the admittance model is given by where Λ Λ Λ d and D d are 6 × 6 positive definite diagonal inertia and damping matrices that can be tuned to obtain a gentle robot behaviour. Using (8) that describes a decoupled dynamic behaviour, the desired velocity vector ϑ ϑ ϑ d is transformed into the incremental motion of desired task x d via a discrete-time integration. Noteworthy, since the installed FT sensor's frame Σ FT is displaced from the Σ EE , the measured wrenches are transformed into Σ EE to obtain the interaction forces, which is then used for the admittance control law.
IV. MOCA-MAN CONTROL FRAMEWORK Fig. 4 illustrates the unified MOCA-MAN control framework. The control flow consists of two modes: the autonomous mode and the conjoined mode.
The autonomous mode is to simulate an action such as manipulation, navigation, etc., which is autonomously executed by the MOCA. An example of the autonomous mode includes that MOCA approaches to the human partner when it is called for a conjoined action using a hand gesture. For this purpose, the position of the human partner is sent to MOCA and a fifth-order polynomial trajectory is calculated as the input to the whole-body impedance controller.
The conjoined mode is aimed to enable the execution of coupled Human-MOCA actions. The examples include comanipulation (e.g., drilling with heavy or vibrating tools), or co-carrying of heavy or potentially dangerous items. In this mode, the weighted dynamically consistent pseudo-inverse in (3), can give a higher priority to the arm or robot movements. For example, if the task requires precise manipulation at a certain position, the controller is able to give a higher priority to the motion of the robotic arm. On the contrary, if the task requires long distance transportation, the mobile base contributes more than the robotic arm to execute the task.
To switch the modes, we use a Myo armband, which integrates eight electromyography sensors, that can provide the recognition of the human gesture. The switching between the autonomous and the conjoined modes is triggered by a double tap hand gesture, which makes MOCA approach to its user. This is followed by the clamping motion, which activates the switch and the conjoined mode, accordingly. Additional hand gestures such as wave in/out and fist are considered to assign higher mobility to the arm/mobile-base movements, and also to open or close the Pisa/IIT Soft-Hand, respectively. The wave in/out gesture assigns different weighting factors to the whole-body Jacobian pseudo-inverse to implement the desired motion of MOCA, giving higher priority either to the arm or to the base motions.
To compensate the unknown weight of the external object held by the MOCA end-effector (SoftHand), in the conjoined mode, the desired stiffness of Cartesian impedance controller is set to a higher value along the z-axis. This feature is activated when the fist gesture is detected, which implies that an object is about to be grasped and picked by the SoftHand.
V. EXPERIMENTS In this section, we first provide experimental validation of the framework's switching capability from autonomous to conjoined mode, where MOCA approaches to its calling partner to make a joint action. We consider two experimental tasks, i.e., long distance carrying and close-proximity comanipulation. Next, we validate the effectiveness of the proposed collaborative framework in conjoined mode through the electromyography (EMG) analysis, where we demonstrate a coherent reduction of five subject's physical efforts while performing a manipulation task.

A. Experimental Setup
During the experiments, the human subjects wore the Myo armband, as well as the wearable part of the developed admittance interface. Instead, the mounting part was attached to the MOCA. The whole-body impedance controller was executed at 1 KHz frequency, and communicated with the gesture recognition module at 10 Hz via ROS.
For the first experiment, MOCA was located 0.6 m far from its human partner, where the target object (a carrying case, 3.1 kg, 440 mm × 340 mm × 140 mm) and the final goal position were located at 0.8 m and 2.8 m distance from the starting position of MOCA, respectively.
To call MOCA and make it autonomously navigate towards the human partner to perform a conjoined action, the subject made a double tap gesture. To determine the position of human, we used an external Optitrack motion tracking system with a single marker set, where was mounted on the wearable part of the admittance interface. The task of conjoined motion was to carry an object to a final goal position that only human was aware of it. The weighted Jacobian and the Cartesian impedance parameters were switched when the human partner made the wave in/out gestures to give high preeminence to the arm movements when reaching to grasp/release the object, and to the mobile base when carrying it. When the given task was accomplished, the human partner released the admittance interface to terminate the conjoined motion.
In a complementary co-manipulation experiment, we compared the human muscle activities of the upper limb, measured by surface EMG sensors, with and without the support of MOCA-MAN. Five volunteers (1 female and 4 male) participated to this experimental session. The task was to perform a screwing operation at 1 m height with a drilling machine (2.5 kg). The task was repeated in two conditions: i) in the conjoined human-MOCA mode where the drill was held and operated by MOCA-MAN, and ii) with the subjects' own hands and without support by MOCA (see also Fig. 5). Six sEMG sensors were placed on the arm of each subject, on the anterior deltoid (AD), the posterior deltoid (PD), the biceps (BC), the triceps (TC), the flexor carpi radialis (FC), the extensor carpi radialis (EC) muscles. The objective was to demonstrate a coherent reduction of the effort across all subjects when MOCA was operated in conjoined mode and assisted its human partner in the execution of the task.

B. Results
The results of the co-carrying experiment are illustrated in Fig. 6. The sequences of the task execution, which are distinguished by the hand gestures, are represented in the first row. The first two plots of Fig. 6 represent the position and orientation of Σ EE w.r.t Σ W . Meanwhile, the third and fourth plots depict the position of Σ EE w.r.t Σ R and position of Σ M w.r.t Σ W , receptively, to show the prioritised arm and mobile base movements due to differentJ W . The bottom plot shows the measured forces at the FT sensor, that was placed between MOCA and the human in the admittance interface, to detect human intended movements.
In the sub-phase (A), MOCA started to move towards the human partner's position when the double tap gesture was recognised. During this phase, MOCA used the mobile base motion to approach to its human partner. Subsequently, then the conjoined action started, as soon as the admittance interface switch was enabled due to the clamping action (sub-phase (B)). In the conjoined action mode, the human partner led the MOCA through the admittance model and via applied interaction forces, as shown in the bottom plot. The desired motion was obtained by (8) When the position of MOCA got closer to the target object by user, the priority motion was switched to the manipulator through the gesture to approach and grasp precisely the object. To carry the object, the robot had to comply with the imposed gravitational forces due to the object weight. As mentioned previously, this feature was activated when the SoftHand performed the grasping action, which was triggered by the fist gesture (sub-phase (C)), hence the parameter of the Cartesian stiffness matrix K along the z-axis increased automatically from 300 N/m to 1800 N/m. Although MOCA, as a supernumerary body, was physically coupled to human, the effect of weight was not transmitted to the human partner, who in turn perceived the physical support. The payload assistance provided by MOCA was also evident in the carrying phase, where the mobile base had a higher priority during the conjoined movement (sub-phase (D)). The measured interaction forces along the z-axis in this phase remained at 2.71 ± 0.97 N until the SoftHand released the object, which occurred in the sub-phase (E). In the sub-phase (F), the human partner released the admittance interface to terminate the conjoined mode. The detachment force F safe = 39.06 ± 2.39 N, and moment T safe = 2.1 ± 0.1 Nm, values were consistent with the safety considerations we took into account when designing the magnetic interface. Table I shows that the overall sEMG results of the manipulation task which confirm the reduction of the muscle activity when MOCA-MAN was used for the screwing task. The decrement ratios across the five subjects were: 49.42±4.23% in the AD, 22  The trajectory of the whole-body motion during the execution of the conjoined action (first two plots), and the separated trajectories of the robotic arm and the mobile base (third and fourth plot, respectively). The dashed lines represent desired motion, and the solid ones represent the measured motion. The measured interaction forces through the admittance interface are illustrated in the last row.
task with and without MOCA) were tested with post-hoc t-tests, where the level of statistical significance was .05.
The measured muscle activity of the AD, BC, TC, FC, and EC showed a significant difference between performing the task with MOCA and without it. The differences of those muscles were: 36.4 ± 4.57% (p ≤ .05) in the AD, 42.6 ± 2.41% (p ≤ .05) in the BC, 33.0 ± 5.31% (p ≤ .05) in the TC, 32.8 ± 4.01% (p ≤ .05) in the FC and 46.6 ± 3.40% (p ≤ .05) in the EC. On the other hand, the difference of muscle activity in the PD was statistically insignificant. The difference was 12.2 ± 4.16% (p = .13). The muscle activity of the PD slightly reduced but not as much as in the other muscles. Nevertheless, the overall effort among subjects decreased significantly during support phase.

VI. CONCLUSION
This paper presented a novel collaborative framework that incorporates the advantages of collaborative robots and supernumerary limbs. An admittance interface was developed to transfer human-intended forces to desired MOCA endeffector trajectories, which was then executed by a wholebody impedance controller. A hand gesture recognition system was implemented to enable the switching between the two MOCA operation modes (autonomous and conjoined action) and to provide the user with the ability to assign a high priority to the arm or the mobile base movements in different phases of the task. We experimentally validated the presented framework in an object handling and a co-manipulation task on a number of subjects. The results demonstrated the high potential of the MOCA-MAN in the successful execution of the joint tasks, and provided solid evidence on a coherent reduction on the effort of the human partners.
Future works will focus on the use of this framework to transfer skills from human partners to MOCA during the execution of complex manipulation tasks.