Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity

This dataset contains evaluation data of the paper "Reinforcement Learning Control of a Biomechanical Model of the Upper Extremity".

Motivation

We address the question whether the assumptions of signal-dependent and constant motor noise in a full skeletal model of the human upper body, together with the objective of movement time minimization, can predict reaching movements.

For evaluation of the learned policy, two tasks are defined in the paper: a Fitts' Law type task and an elliptic via-point task.

General description of the dataset

• Fitts' Law Type Task
  • This dataset incorporates detailed information of all 6500 synthesized movements generated in the Fitts' Law type task (following the ISO 9241-9 standard).

  • For each of the 10 task conditions differing in distance between targets ("dist<xxx>" in filename)
    and ID ('ID<xxx>' in filename), there are two .csv-files:
    - one with detailed trajectory information on a sample-to-sample basis ("ISO_SAMPLES" in filename), and
    - one with aggregated movement information on an episode basis ("ISO_METRICS" in filename).

  • In addition, for each task condition and each of the 13 movement directions in the Fitts' Law type task,
    we include 6 figures: Position, Velocity, and Acceleration Profiles, as well as 3D movement path, Phasespace, and Hooke plots.
    Apart from the 3D plots, all figures use centroid projections of the respective trajectory onto the vector between initial and target position.
    The first integer in the file name denotes the movement direction number, starting with "0" for movements between the targets 1 and 2, "1" for movements between the targets 2 and 3 etc.
    The file "6_distance0.35_ID2_policy2100000_phasespace.png", e.g., shows velocity plotted againt position for all 50 movements between the targets 7 and 8 in the task condition with ID 2 and 35cm diameter of the target circle (see Fig 2 in Paper).

• Elliptic Task

  • This dataset also contains two CSV-files with data of the trajectory generated by the final policy in the elliptic task:
    - one with detailed trajectory information on a sample-to-sample basis ("ELLIPSE_SAMPLES" in filename), and
    - one with aggregated movement information on an episode basis, where a new episode starts
    whenever the target on the ellipse given to the policy switches ("ELLIPSE_METRICS" in filename).

Description of the .csv-files

• SAMPLES Files
  • "time": time after reaching the initial target (target 1 in Fig 2) for the first time (in seconds)
  • "elv_angle_pos" - "flexion_pos": angle of respective independent DOF (in radians) *
  • "elv_angle_vel" - "flexion_vel": angular velocity of respective independent DOF (in radians/s) *
  • "end-effector_xpos_x" - "end-effector_xpos_z": 3D position of end-effector in global coordinates (in meters) *
  • "target_xpos_x" - "target_xpos_z": 3D position of target sphere in global coordinates (in meters) *
  • "end-effector_xvelp_x" - "end-effector_xvelp_z": positional velocities of end-effector (in meters/s) *
  • "target_xvelp_x" - "target_xvelp_z": positional velocities of target sphere (in meters/s) *
  • "accsensor_end-effector_x" - "accsensor_end-effector_z": positional acceleration of end-effector (in meters/s^2) *
  • "E_elv_angle" - "E_flexion": activation of respective independent DOF *
  • "E_elv_angle_derivative" - "E_flexion_derivative": derivative of activation of respective independent DOF *
  • "difference_vec_x" - "difference_vec_z": vector between the end-effector attached to the index finger and the target, pointing towards the target (in meters) *
  • "centroid_vel_projection": projection of end-effector velocity towards target (in meters/s) *
  • "target_width": radius (!) of the target sphere (in meters) *
  • "A_elv_angle" - "A_flexion": action vector
  • "thorax_tx_frc" - "wrist_hand_r3_frc": net external force at respective DOF (including dependent and fixed DOFs such as "thorax_tx" (thorax translation))
  • "reward": reward obtained in this step
  • "step_type": 0=initial step of episode, 1=intermediate step of episode, 2=terminal step of episode
  • "target_switch": whether the target switched in this step
  • "discount": internal value of tf-agents (does not correspond to the discount factor gamma, which is additionally applied!)
  • "thorax_tx_pos" - "wrist_hand_r3_pos": angle of respective dependent DOF (in radians)
  • "thorax_tx_vel" - "wrist_hand_r3_vel": angular velocity of respective dependent DOF (in radians)
• METRICS Files
  • Index: episode ID
  • "Init_X" - "Init_Z": initial position in global coordinates (in meters)
  • "Init_X" - "Init_Z": target position in global coordinates (in meters)
  • "Init_Distance": distance between last target (i.e., desired initial position) and current target (in meters)
  • "Target_Width_Diameter": target width diameter (in meters)
  • "Movement_ID": Index of Difficulty of current movement (using the Shannon Formulation) (in bits)
  • "target_accuracy": 1 - (<remaining distance to target center at the end of the episode>/<target radius>) if end-effector is inside target, 0 else
  • "movement_time": duration of the episode (in seconds)
  • "episode_successful": whether episode terminated successfully within the permitted 1.5 seconds
  • "dist2target": remaining distance to target center at the end of the episode (in meters)

-------------------------------------
* included in state space