Estimation of Phantom Arm Mechanics About Four Degrees of Freedom After Targeted Muscle Reinnervation

The intuitive control of bionic arms requires estimation of amputee’s phantom arm movements from residual muscle bio-electric signals. The functional use of myoelectric arms relies on the ability of controlling large sets of degrees of freedom (>3 DOFs) spanning elbow, forearm, and wrist joints. This would assure optimal hand orientation in any environment. As part of this paper we recorded high-density electromyograms with >190 electrodes from the residual stump of a trans-humeral amputee who underwent targeted muscle reinnervation. We employed clustering to determine eight spatially separated sub-sets of channels sampling electromyograms associated to the actuation of four phantom arm DOFs. We created a large-scale musculoskeletal model of the phantom arm encompassing 33 musculo-tendon units. For the first time, this enabled the accurate electromyography-driven simulation of complex phantom joint rotations about elbow flexion–extension, forearm pronation–supination, wrist flexion–extension, and radial–ulnar deviation. These results support the potential for a new class of bionic limbs that are controlled as natural extensions of the body, an important step toward next-generation prosthetics that mimic human biological functionality and robustness.

Elbow-wrist control is a major target in healthcare roadmaps [9]. The upper extremity relies on the hand, demanding simultaneous elbow-wrist orientation and stability [10]. This requires estimating phantom arm movements from muscle bio-electric signals (electromyograms, EMGs) and controlling bionic limbs that mimic the intended motion [11]. Current machine learning methods based on pattern recognition or regression directly map EMGs into joint angles [12]. However, mappings learned in a condition (e.g., low loads) may not generalize to unseen conditions (e.g., high loads), hampering control robustness. This contributes to the unsatisfactory clinical impact of current bionic arms [13], [14]. The major barrier to progress is the limited understanding of the neuro-mechanics of biological limb movements [2], [15].
To make natural joint rotations, the central nervous system (CNS) interacts with the musculoskeletal system [10], [16]. Despite theoretical knowledge, there is no framework that synthetizes these processes into bionic limb technologies.
In this work we recorded muscle high-density (HD)-EMGs from large sets of electrodes and reconstructed the non-linear transformations that lead to the production of musculoskeletal forces in the phantom upper extremity. This method is based on a biomimetic model-based decoder, i.e., a computational neuro-mechanical model that explicitly synthesizes the dynamics of the musculoskeletal system as controlled by HD-EMG-derived muscle activation signals (Fig. 1).
We recently showed for the first time the possibility of using an EMG-driven musculoskeletal modelling approach to estimate phantom wrist-hand moments about two degrees of freedom (DOFs) in a transradial amputee and to enable realtime prosthesis control [17]. We here extend this model-based approach to a trans-humeral amputee who underwent the targeted muscle reinnervation surgery (TMR) for the estimation of phantom limb mechanics about four DOFs.
Current state of the art work on upper limb amputee's musculoskeletal modelling proposed formulations based on intact-limbed individuals about a single joint DOF (i.e., elbow flexion-extension) [18] and two-DOF [19]. A simplified lumped-parameter model of the hand [20], [21] was used to compute wrist and metacarpophalangeal joint flexion/extension angles in a transradial amputee [20]- [22].
Our proposed work provides the first quantitative results showing that neuro-mechanical modelling in conjunction with HD-EMG can be successfully used to decode the mechanics of the phantom elbow, forearm and wrist across four DOFs. (Left) High-density electromyograms (HD-EMG) are recorded using bi-dimensional grids from the amputee's residual muscle tissues after targeted muscle reinnervation. HD-EMGs are post-processed via filtering and normalization (Section II). Clustering is applied to determine sub-sets of channels associated to the control of individual degrees of freedoms (DOFs, Section II, Fig. 2). Clustered EMGs are used to extract the neural activations produced by the human central nervous system in the control of the phantom limb (Section II-D). (Center) A subjectspecific musculoskeletal geometry model is built for the transhumeral amputee. The model incorporates a total of 33 wire-like musculotendon units (MTU) and MTU-to-bone wrapping surfaces on both intact and phantom limbs. (Right) Phantom limb MTU forces are simultaneously projected, via moment arms, to four DOFs in the phantom arm including elbow flexion-extension (EFE), the forearm pronation-supination (FPS), the wrist flexion-extension (WFE), and radial-ulnar deviation (RUD). Moments predicted in the phantom limb are to be used as control command for bionic limbs. These reconstruct phantom limb mechanics in the real-world intuitively.
The results we present can advance the field of mindcontrolled bionic limbs by enabling mimicking the mechanical function of the amputee's lost biological extremities. We present and discuss how this new procedure can be used to establish intuitive human-machine interfaces (HMIs).

A. Subject
The University Medical Center Göttingen Ethical Committee approved all experimental procedures (approval numbers 9/2/12 and 11/10/14). Procedures were conducted in accordance with the Declaration of Helsinki.
One individual (age: 51 years, weight: 82 kg; height: 172 cm) with left arm transhumeral amputation volunteered for this investigation and provided signed informed consent. The amputee underwent TMR surgery four years and four months before the experiment [23]. This resulted in the medianus, ulnaris, and radialis nerves being reinnervated into the brachialis, caput breve bicipitis, and caput laterale tricipitis muscles, respectively.

B. Data Collection
We recorded surface HD-EMG signals using three bi-dimensional electrode grids (ELSCH064NM3, OT Bioelettronica, IT) mounted around the residual limb. The 8×8 grids had an inter-electrode distance of 10 mm and were located in correspondence of the residual upper arm frontal (grid 1), lateral (grid 2), and posterior compartments (grid 3) to cover all targeted reinnervation sites. The three electrode grids were connected to a 256-channel EMG amplifier (EMGUSB2, OT Bioelettronica). The recorded HD-EMG signals were band-pass filtered between 3-900 Hz, and A/D converted using a 12-bit converter at a sampling rate of 2048 Hz.
Each individual was initially asked to perform maximal voluntary contractions articulating elbow and wrist flexionextension, radial-ulnar deviation, and forearm pronationsupination. To derive signal envelopes, EMGs were highpass filtered (30Hz), full-wave rectified, and low-pass filtered (6 Hz) using a second-order Butterworth filter. Resulting peak-processed values were used for the subsequent EMG normalization during the neuro-mechanical modelling tests (Section II-D).
High-speed motion capture data were collected synchronously to HD-EMGs. Upper limb kinematics was recorded (256 Hz) using a seven-camera system (Qualisys, Göteborg, Sweden) and a set of 18 retro-reflective markers placed on the individual's intact right upper extremity, residual left upper extremity, trunk, and pelvis.

C. Experimental Procedures
Data were recorded during one static anatomical pose and mirrored dynamic bilateral contractions that articulated the elbow flexion-extension (EFE), the forearm pronationsupination (FPS), the wrist flexion-extension (WFE), and radial-ulnar deviation (RUD) DOFs, both in the intact and phantom limbs.
Dynamic contractions were controlled using metronomic acoustic cues at 1 beat per second. Starting from neutral position, the subject performed elbow flexion, forearm pronation, wrist extension and ulnar-deviation followed by elbow extension, forearm supination, and wrist flexion and radial deviation to reach the starting neutral position. A total of six repetitions Intact and phantom limb joint moments similarity was quantified using the Pearson product moment correlation coefficient (R) and the root mean squared difference (RMSD).

D. Neuro-Mechanical Modelling
We used the open-source software OpenSim [25] to scale a generic upper extremity model of the musculoskeletal geometry [26], [27] to match the subject's anthropometry. The musculoskeletal geometry model had seven upper extremity DOFs and incorporated a total of 33 muscle-tendon units (MTUs), spanning the shoulder, elbow, wrist and hand joints (Table I).
1) Model Scaling and Inverse Dynamics: During the scaling process virtual markers were placed on the generic musculoskeletal geometry model based on the position of the experimental markers from the static pose. The model anthropomorphic properties as well as MTU insertion, origin and MTU-to-bone wrapping points were linearly scaled on the basis of the relative distances between experimental and corresponding virtual markers [25]. Inverse kinematics solved for three-dimensional joint angles that minimized the least-squared error between experimental and virtual marker locations during dynamic trials [28]. The generated kinematics was then used to obtain dynamically consistent joint moments via residual reduction analysis [29], i.e., joint moments reconstructing experimental joint angles when driving forward dynamics arm simulations [29]. We call these the "experimental joint moments". The alternative pathway to joint moments was by HD-EMG-driven modelling or neuro-mechanical modeling [30].
2) HD-EMG Clustering: The HD-EMG bio-electric activity of muscle fibers in the TMR individual's stump was clustered according to the DOFs it contributed to actuate (Table I, Fig. 2). Clustering was obtained from muscle contractions associated to the control of individual DOFs. A total of six trials were performed for each DOF. For each contraction, the steady-state portion of the EMG signal was used for analysis. In this, an 80% threshold on normalized EMG linear envelopes across all channels was applied. The threshold level was iteratively identified as the level resulting in least channel activity overlap across DOFs. A grid channel was included within a cluster if it displayed above-threshold normalized activity across all six trials.
Above-threshold normalized envelopes were converted into neural activations using a twitch model based on a time-history dependent recursive filter and a non-linear transfer function as previously proposed by Milner-Brown et al. [31], Manal and Buchanan [32], and Sartori et al. [33].
3) EMG-Driven Forward Dynamics: The resulting neural activations were prescribed to sets of MTUs having moment arms about the selected DOFs (Table I). Each MTU received the mean of the neural activations mapped to the MTU-spanned DOFs. Experimental joint angles were used as input to multidimensional cubic B-splines that synthetized the OpenSim subject-specific MTU geometry and computed the resulting MTU length and moment arms as previously described [34]. Neural activations and MTU length were used to control a Hill-type muscle model and estimate instantaneous length, contraction velocity, and force in the muscle fibers, and strain and force in the series-elastic tendon within each MTU [35], [36]. MTU forces were projected onto all upper extremity DOFs simultaneously via B-spline computed MTU moment arms.

4) Model Calibration:
The neuro-mechanical model was calibrated to convert neural activations to individual MTU force and account for physiological and force-generating differences across individuals. The model calibration received three input signals including: neural activations as well as experimental joint angles and moments derived from the subject's intact upper arm during one bi-lateral mirror movement.
MTU-specific parameters were adjusted within physiological boundaries to minimize the sum of the mean square differences between the predicted phantom limb [17]. Calibrations and simulations were conducted on a laptop computer with an i7 Intel Processor and 16 GB RAM. Fig. 2. Channel clustering in high-density grids across four degrees of freedoms (DOFs). The schematics of three 64-channel grids is depicted. Grids were placed on the amputee's stump as described in Section II-B. The bio-electric activity of muscle fibers recorded via these grids was first converted into normalized linear envelopes (see Methods Section). Normalized envelopes were thresholded (80% of normalized amplitude). Above-threshold normalized envelopes are depicted during tasks involving the actuation of individual DOFs. The bio-electrical activity associated to a specific DOF was converted into neural activations using a second-order recursive filter (Section II-D). Resulting activations were then prescribed to all musculotendon units (MTUs) in the computational model having moment arms about the selected DOFs (Table I). Each MTU received the mean of the neural activations mapped to the MTU-spanned DOFs.

III. RESULTS
Clustering of HD-EMGs across the three grids resulted in eight spatially separated channel sub-sets associated to the control of individual directions within each DOFs. A total of 47 channels were selected, which included 27 channels for grid 1, 15 channels for grid 2, and 8 channels for grid 3 (Fig. 2). The eight clustered areas were mapped to the 33 MTUs as in Table I.
The largest level of channel overlapping was observed in the control of elbow-flexion and forearm pronation, for which a total of five channels were shared across the two DOFs. The control of wrist-supination and wrist-extension had four shared channels. The control of forearm-pronation and wrist-flexion as well as forearm pronation and radial-deviation had three shared channels respectively. The control of wrist-supination and elbow extension had one shared channel. No overlapping was observed across other DOF pairs.
With the given clustering, phantom limb joint moment predicted from HD-EMGs (TMR side) well reflected those experimentally measured via residual reduction analysis (see Methods Section II) from the intact limb during bi-lateral mirror tasks. Fig. 3 shows mean and standard deviation for moments measured from the intact limb and for those estimated from the phantom limb. Joint moment similarity metrics were comparable across all DOFs including elbow flexionextension (R = 0.77 ± 0.04, RMSD = 0.65 ± 0.06 Nm), forearm pronation-supination (R = 0.87 ± 0.05, RMSD = 0.026 ± 0.004 Nm), wrist flexion-extension (R = 0.95 ± Phantom arm movements corresponded to EMG patterns, as measured from the stump (Fig. 2), that well reflected those measured from the contralateral intact limb (Fig. 4). When employing the grid channel clustering depicted in Fig. 2 and when processing cluster-specific EMGs as described in Sections II-B and II-D, the resulting activations driving phantom MTUs (Table I) displayed high similarity (R > 0.6) with respect to activations for intact-limb muscles (Fig. 4). Shape similarity between stump-clustered and intact-limb EMGs were 0.71 ≤ R ≤ 0.94 for EFE-actuating muscles, 0.71 ≤ R ≤ 0.99 for WFE-actuating muscles, and 0.83 ≤ R ≤ 0.97 for RUD-actuating muscles. For FPS-actuating muscles, the R-coefficient was in the range 0.61 ≤ R ≤ 0.94 for muscles including extensor digitorum, flexor/extensor carpi radialis, extensor carpi ulnaris. It was unfavorable (−0.13 ≤ R ≤ 0.54) for biceps brachii, flexor carpi ulnaris, and flexor digitorum.
Calibration time of individual DOFs ranged between 7 minutes (elbow flexion-extension) and 11 minutes (radial-ulnar deviation). Calibration of four DOFs was completed within 45 minutes.

IV. DISCUSSION
Man-machine interfaces for bionic limbs consist in connecting the amputee's nervous system with external robotic artificial limbs. Although brain interfacing is possible [37], peripheral muscle interfaces represent the only clinically viable approach for bionic arms to date [33], [38]. After TMR surgeries, muscles can be considered as biological amplifiers of nerve activity and represent the optimal level for establishing interfaces [39].
With this work, we established an interface with the amputee's neuromuscular system in vivo via recordings of high-density surface electromyograms using three 64-channel grids [40]. We applied a multi-channel clustering and a thresholding technique that minimized overlaps in the myoelectric activity associated to the control of four selected DOFs (Fig. 2). This enabled deriving DOF-specific EMG activity that could be used to simulate the neural activation sent to large groups of synergistic muscles (Table I).
We then reconstructed all the transformations that took place from the onset of multi-muscle activation to the production of musculo-skeletal forces in the amputee's phantom arm. This was achieved by amputee-specific neuro-mechanical modelling (Fig. 1) [30], [41]. Within this framework, we proved the possibility of using clustered HD-EMGs for the blinded prediction of mechanical moments across four DOFs spanning the phantom elbow, forearm and wrist.
Results showed that stump-measured clustered EMG activity well reflected EMG patterns recorded during mirror tasks from intact-limb muscles (Fig. 4). Shape similarity was only unfavourable (R < 0.6) for three muscles actuating FPSrotations. That was due to the fact that prime FPS muscles (i.e., pronator teres and quadratus, supinator muscle), could not be measured via surface EMGs from the intact-side. Therefore, comparison only involved secondary FPS muscles, which also contributed to EFE, WFE and RUD rotations (Fig. 4). Moreover, tasks were performed with no feedback phantom arm position and orientation, which prevented the amputee to precisely track intact-limb motions. Overall, these results suggest that our proposed clustering and modelling procedure resulted in realistic activations and moment controlling a total of four phantom joint rotations (Figs 3-4).
In a recent paper, we demonstrated the possibility of employing EMG-driven musculoskeletal modelling for the online control of a prosthetic hand. This was based on a smaller-scale musculoskeletal model for the online control of pronation-supination and hand opening-closing moments in a transradial amputee. To the best of our knowledge, the present study is the first that shows the ability of recording HD-EMG activity from more than 190 channels, simulating the mechanics of large-scaled models comprising more than 30 MTUs, and estimating the resulting mechanical moments actuating four DOFs in phantom arm movements. Whether an EMG clustering method could be established to extract patterns similar to those derived from intact-limb EMGs and whether large-scale models could be established to predict realistic moments about four phantom DOFs (three of which in the same joint, the wrist) represented major aspects not addressed before.
The proposed approach has the potential of reducing the space of feasible solutions in mapping electromyograms into intended motions, with respect to signal-based techniques, potentially enhancing prosthesis control robustness [17]. This provides benefits with respect to model-free techniques that may potentially map input EMGs into joint positions outside of human body feasible ranges [30].
This study has some limitations. Although the results demonstrated accuracy in estimating four-DOF mechanical moments in the amputee's phantom arm, these do not allow directly assessing control proportionality of a real prosthetic limb. Systematic work must be conducted to assess performance during online prosthetic control. However, the authors' recent work [17] indicated that the ability of estimating both shape and amplitude of phantom joint moments is crucial for online prosthetic control. Therefore, this study results (Figs 3-4) suggest online control feasibility.
The proposed methods are not immediately transferrable to clinical practice. This is due to use of non-stretchable and wired HD-EMG-grids requiring some preparation time and potentially constraining patient's movements. While the aim of this study was to solely assess the ability of estimating 4-DOF-phantom limb mechanics via HD-EMG and modelling, future work will investigate short-term and mid-term solutions of clinical viability.
Short-term solutions will rely on channel reduction techniques. HD-EMG clustering results indicated that the initial 192-channel set could be reduced to a subset of 47 channels (Fig. 2), which is relevant for clinical translation. Future work will assess the possibility of employing our proposed clustering method in conjunction with dimensionality reduction techniques [30] for determining minimal optimal sets of bipolar electrodes.
Mid-term solutions will rely on advances in wireless HD-EMG [42] as well as stretchable-electronics [43], [44] and epidermal electronics [45] for EMG recording. This will relax the need for channel-reduction and provide generalizable solutions. In this context, the favorable channel spatial separation identified in this study may depend on how the TMR surgery was performed. As this level of spatial separation cannot be reached for all patients, highly dense sensing methods for EMGs are desired as they would provide highresolution sampling across surgeries. Moreover, HD-EMG would allow for the interferent EMG decomposition into constituent motor unit discharges, thereby potentially enabling further performance improvement in phantom limb mechanics estimation, as previously suggested [39]. Future work will systematically assess to what extend HD-EMG decomposition can improve our methodology performance.
Future work will also investigate the possibility of operating the proposed large-scale model in real-time for the concurrent control of a prosthetic limb with active elbow and wrist motors. To further assess generalizability and robustness of the proposed technique, the current methods will be applied on a larger patient population with different levels of amputations and TMR surgeries.

V. CONCLUSION
We proposed a model-based technique that mimics how CNS surrogates (Fig. 4, red thick line) control phantom arms mechanics (Fig. 3) in a TMR individual with trans-humeral amputation. For the first time, our proposed methodology combines high-density EMG and large-scale musculoskeletal modelling to estimate four-DOF arm mechanics based on accurate electromyography-driven simulations of phantom limb movements. This approach reduces the space of feasible solutions in mapping electromyograms into intended motions, with respect to model-based techniques, potentially enhancing prosthesis control robustness. This biomimetic technique may have substantial clinical benefits over conventional non-biomimetic machine learning methods [46] as it may contribute to a greater sense of ownership over the replacement body part and to a greater adoption of advanced robotic prostheses in daily lives [30].