JointTracker: Real-time inertial kinematic chain tracking with joint position estimation
Authors/Creators
- 1. IT & Engineering, International University of Applied Sciences, Erfurt, Thuringia, 99084, Germany
- 2. Augmented Vision, German Research Center for Artificial Intelligence, Kaiserslautern, Rhineland-Palatinate, 67663, Germany
Description
In-field human motion capture (HMC) is drawing increasing attention due to the multitude of application areas. Plenty of research is currently invested in camera-based (markerless) HMC, with the advantage of no infrastructure being required on the body, and additional context information being available from the surroundings. However, the inherent drawbacks of camera-based approaches are the limited field of view and occlusions. In contrast, inertial HMC (IHMC) does not suffer from occlusions, thus being a promising approach for capturing human motion outside the laboratory. However, one major challenge of such methods is the necessity of spatial registration. Typically, during a predefined calibration sequence, the orientation and location of each inertial sensor are registered with respect to the underlying skeleton model. This work contributes to calibration-free IHMC, as it proposes a recursive estimator for the simultaneous online estimation of all sensor poses and joint positions of a kinematic chain model like the human skeleton. The full derivation from an optimization objective is provided. The approach can directly be applied to a synchronized data stream from a body-mounted inertial sensor network. Successful evaluations are demonstrated on noisy simulated data from a three-link chain, real lower-body walking data from 25 young, healthy persons, and walking data captured from a humanoid robot. The estimated and derived quantities, global and relative sensor orientations, joint positions, and segment lengths can be exploited for human motion analysis and anthropometric measurements, as well as in the context of hybrid markerless visual-inertial HMC.
Files
openreseurope-4-18306.pdf
Files
(5.6 MB)
| Name | Size | Download all |
|---|---|---|
|
md5:c2b99f3a35e2e08c906d433ec9927fa3
|
5.6 MB | Preview Download |
Additional details
References
- Colyer SL, Evans M, Cosker DP (2018). A Review of the Evolution of Vision-Based Motion Analysis and the Integration of Advanced Computer Vision Methods Towards Developing a Markerless System. Sports Med Open. doi:10.1186/s40798-018-0139-y
- Yamamoto M, Shimatani K, Ishige Y (2022). Verification of gait analysis method fusing camera-based pose estimation and an IMU sensor in various gait conditions. Sci Rep. doi:10.1038/s41598-022-22246-5
- Tagliamonte NL, Peruzzi A, Accoto D (2014). Assessment of lower limbs kinematics during human-robot interaction using inertial measurement unit. Gait & Posture. doi:10.1016/j.gaitpost.2014.05.050
- Bleser G, Damen D, Behera A (2015). Cognitive Learning, Monitoring and Assistance of Industrial Workflows Using Egocentric Sensor Networks. PLoS One. doi:10.1371/journal.pone.0127769
- Wong C, Zhang ZQ, Lo B (2015). Wearable sensing for solid biomechanics: A review. IEEE Sens J. doi:10.1109/JSEN.2015.2393883
- Menolotto M, Komaris DS, Tedesco S (2020). Motion capture technology in industrial applications: A systematic review. Sensors (Basel). doi:10.3390/s20195687
- (null). Vicon.
- (null). Qualisys.
- (null). Optitrack.
- (null). Theia3D.
- Cao Z, Hidalgo G, Simon T (2021). Openpose: Realtime multi-person 2d pose estimation using part affinity fields. IEEE Trans Pattern Anal Mach Intell. doi:10.1109/TPAMI.2019.2929257
- Rempe D, Birdal T, Hertzmann A (2021). HuMoR: 3D Human Motion Model for Robust Pose Estimation. doi:10.1109/ICCV48922.2021.01129
- Zou S, Guo C, Zuo X (2021). EventHPE: Event-Based 3D Human Pose and Shape Estimation. doi:10.48550/arXiv.2108.06819
- Tome D, Alldieck T, Peluse P (2023). Selfpose: 3d egocentric pose estimation from a headset mounted camera. IEEE Trans Pattern Anal Mach Intell. doi:10.1109/TPAMI.2020.3029700
- Lin TY, Maire M, Belongie S (2015). Microsoft COCO: Common Objects in Context. doi:10.48550/arXiv.1405.0312
- Mahmood N, Ghorbani N, Troje NF (2019). AMASS: Archive of Motion Capture As Surface Shapes. doi:10.1109/ICCV.2019.00554
- Artacho B, Savakis A (2022). Unipose+: A unified framework for 2d and 3d human pose estimation in images and videos. IEEE Trans Pattern Anal Mach Intell. doi:10.1109/TPAMI.2021.3124736
- Artacho B, Savakis A (2020). UniPose: Unified Human Pose Estimation in Single Images and Videos. doi:10.1109/CVPR42600.2020.00706
- Xu Y, Zhang J, Zhang Q (2022). ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation.
- Martinez J, Hossain R, Romero J (2017). A simple yet effective baseline for 3d human pose estimation. doi:10.1109/ICCV.2017.288
- (null). Xenoma.
- Xiao X, Zarar S (2018). Machine Learning for Placement-Insensitive Inertial Motion Capture. doi:10.1109/ICRA.2018.8463176
- Lorenz M, Bleser G, Akiyama T (2022). Towards artefact aware human motion capture using inertial sensors integrated into loose clothing. doi:10.1109/ICRA46639.2022.9811933
- Roetenberg D, Luinge H, Slycke P (2009). Xsens MVN: Full 6DOF Human Motion Tracking Using Miniature Inertial Sensors.
- Harle R (2013). A survey of indoor inertial positioning systems for pedestrians. IEEE Communications Surveys and Tutorials. doi:10.1109/SURV.2012.121912.00075
- Ligorio G, Sabatini AM (2016). Dealing with magnetic disturbances in human motion capture: A survey of techniques. Micromachines (Basel). doi:10.3390/mi7030043
- Miezal M, Taetz B, Bleser G (2016). On inertial body tracking in the presence of model calibration errors. Sensors (Basel). doi:10.3390/s16071132
- Taetz B, Bleser G, Miezal M (2016). Towards self-calibrating inertial body motion capture.
- Zimmermann T, Taetz B, Bleser G (2018). Imu-to-segment assignment and orientation alignment for the lower body using deep learning. Sensors (Basel). doi:10.3390/s18010302
- Pacher L, Chatellier C, Vauzelle R (2020). Sensor-to-Segment Calibration Methodologies for Lower-Body Kinematic Analysis with Inertial Sensors: A Systematic Review. Sensors (Basel). doi:10.3390/s20113322
- Vitali RV, Perkins NC (2020). Determining anatomical frames via inertial motion capture: A survey of methods. J Biomech. doi:10.1016/j.jbiomech.2020.109832
- Seel T, Raisch J, Schauer T (2014). IMU-based joint angle measurement for gait analysis. Sensors (Basel). doi:10.3390/s140406891
- Laidig D, Weygers I, Seel T (2022). Self-Calibrating Magnetometer-Free Inertial Motion Tracking of 2-DoF Joints. Sensors (Basel). doi:10.3390/s22249850
- McGrath T, Stirling L (2020). Body-Worn IMU Human Skeletal Pose Estimation Using a Factor Graph-Based Optimization Framework. Sensors (Basel). doi:10.3390/s20236887
- McGrath T, Stirling L (2022). Body-Worn IMU-Based Human Hip and Knee Kinematics Estimation during Treadmill Walking. Sensors (Basel). doi:10.3390/s22072544
- Rajagopal A, Dembia CL, DeMers MS (2016). Full-Body Musculoskeletal Model for Muscle-Driven Simulation of Human Gait. IEEE Trans Biomed Eng. doi:10.1109/TBME.2016.2586891
- Seel T, Schauer T, Raisch R (2012). Joint axis and position estimation from inertial measurement data by exploiting kinematic constraints. doi:10.1109/CCA.2012.6402423
- Olsson F, Halvorsen K (2017). Experimental evaluation of joint position estimation using inertial sensors. doi:10.23919/ICIF.2017.8009669
- Markley FL, Crassidis JL (2014). Fundamentals of Spacecraft Attitude Determination and Control.
- Kok M (2014). Probabilistic modeling for positioning applications using inertial sensors.
- Humpherys J, Redd P, West JM (2012). A fresh look at the kalman filter. SIAM Rev. doi:10.1137/100799666
- Skoglund MA, Hendeby G, Axehill D (2015). Extended kalman filter modifications based on an optimization view point.
- de Vries WHK, Veeger HEJ, Baten CTM (2009). Magnetic distortion in motion labs, implications for validating inertial magnetic sensors. Gait Posture. doi:10.1016/j.gaitpost.2008.12.004
- Skog I, Händel P, Nilsson JO (2010). Zero-velocity detection—an algorithm evaluation. IEEE Trans Biomed Eng. doi:10.1109/TBME.2010.2060723
- Teufl W, Miezal M, Taetz B (2018). Validity, test-retest reliability and longterm stability of magnetometer free inertial sensor based 3d joint kinematics. Sensors (Basel). doi:10.3390/s18071980
- Teufl W, Lorenz M, Miezal M (2019). Towards inertial sensor based mobile gait analysis: Event-detection and spatio-temporal parameters. Sensors (Basel). doi:10.3390/s19010038
- Kok M, Hol JD, Schön TB (2014). An optimization-based approach to human body motion capture using inertial sensors. IFAC Proceedings. doi:10.3182/20140824-6-ZA-1003.02252
- Black HD (1964). A passive system for determining the attitude of a satellite. AIAA Journal. doi:10.2514/3.2555
- Raue A, Becker V, Klingmüller U (2010). Identifiability and observability analysis for experimental design in nonlinear dynamical models. Chaos. doi:10.1063/1.3528102
- Triggs B, McLauchlan PF, Hartley RI (2000). Bundle adjustment - a modern synthesis.
- Kok M, Eckhoff K, Weygers I (2021). Observability of the relative motion from inertial data in kinematic chains. doi:10.48550/arXiv.2102.02675
- Casella G, Berger RL (2002). Statistical inference.
- Ludbrook J (2010). Linear regression analysis for comparing two measurers or methods of measurement: But which regression?. Clin Exp Pharmacol Physiol. doi:10.1111/j.1440-1681.2010.05376.x
- Teufl W, Miezal M, Taetz B (2023). Zenodo.
- Kainz H, Carty CP, Modenese L (2015). Lloyd. Estimation of the hip joint centre in human motion analysis: A systematic review. Clin Biomech (Bristol, Avon). doi:10.1016/j.clinbiomech.2015.02.005
- Lorenz M, Aller F, Bleser-Taetz G (2023). Zenodo.
- Olsson F, Kok M, Seel T (2020). Robust Plug-and-Play Joint Axis Estimation Using Inertial Sensors. Sensors (Basel). doi:10.3390/s20123534
- Bouvier B, Duprey S, Claudon L (2015). Upper Limb Kinematics Using Inertial and Magnetic Sensors: Comparison of Sensor-to-Segment Calibrations. Sensors (Basel). doi:10.3390/s150818813
- Lorenz M, Taetz B, Bleser G (2020). An approach to magnetometer-free on-body inertial sensors network alignment. IFAC-PapersOnLine. doi:10.1016/j.ifacol.2020.12.393