Modular System Design Approach for Online Ergonomics Assessment in Agile Production Environment*

Shorter product lifecycles, mass individualization and agile concepts for lot size1 require more flexible production. Constant challenges may result in optimized processes. However, opimization effects not only on the production technologies but also on the operators and their ergonomics. A simulation based ergonomic evaluation in such hybrid workspaces is tough and fruitless due to constantly changing in the production processes and to the diversity of the human activities. An online ergonomics assessment and real-time feedback ensures that employees improve the quality of their workplace and maintain healthy working conditions. This work proposes a method for agile production or traditional factory floors using a marker less camera-based system using real-time joint position estimations as input and providing a continuous or summarized individual ergonomic feedback as output. Furthermore, first implementation results considering the requirements and overall system design for online ergonomic evaluation in agile production environment are discussed.


I. INTRODUCTION
Musculoskeletal disorders (MSDs) are still the main reason for incapacity to work. Especially, the work-related musculoskeletal disorders (WRULDs) are causing high costs for enterprises and society [1].
Benefits of good ergonomics and health-promoting workplace design [2]:  Reduction of absenteeism and the risk of accidents  Intervention for more well-being  Higher productivity and motivation of employees  Increase of employee loyalty  Improvement of the product quality  Better planning and future security  Cost saving and avoidance Conventional way to conduct ergonomic assessment is to use pen & paper-based assessment methods. In most of the methods, the calculation of point values makes it possible to estimate the probability of health damage. One of the most comprehensive and widely used ergonomic evaluation method is the Ergonomic Assessment Work-Sheet (EAWS). working methods and ergonomic risk assessment. The method is used in industrial manufacturing. [3]. The method includes all usual manual material handling activities.
Subjective observation and manual assessment using pen & paper-based assessment carries the risk of low accuracy and high intra-and inter-observer variability. Common Pen & paper methods are frequently integrated in simulation systems [4]. The use of simulation tools and digital twins could lack of accuracy and generic system parameterization. Furthermore, it is very hard to include all possible human activities and processes in agile environment [5], [6]. This could lead to inaccurate and fruitless evaluation results. In addition, the pre-processing of specific data is usually timeconsuming.
Dynamic workplaces and environments are driven by Industry 4.0. The quest for mass-produced individual products requires a high degree of flexibility and adaptability. In most of today's industrial processes, complete predictability and/or compliance with detailed, predetermined work procedures is unattainable and incompatible with the quest for efficiency. Therefore, there is an increasing demand for (partially) autonomous and adaptive production processes. Even in times of Industry 4.0 and smart automation solutions, man, with his manpower and abilities as a highly flexible player, plays a key role. In the factory of the future, man works side by side with smart intelligent robots -both forming a highly efficient symbiosis. The collaborative use of robots especially allows:  Taking over forces and loads  Support of the posture  Reduction of walking distances Changes in the world of work will tend to increase the cognitive workload of workers and change the relationship between physical and cognitive workload [7]. On the other hand, these changes will lead to greater job enrichment and increased autonomy for the workers [8]. New production paradigms require new planning and engineering methods, including occupational health and safety. Ergonomics design and assessment of human activities in flexible, high dynamic production environment is challenging. Conventional offline ergonomics evaluation methods are not designed for shortterm changes in production processes. Research and development of socio-technical systems evaluating ergonomics during operation and contribute to improving ergonomics are needed. In the present work, requirements for such a system are identified and system design is proposed, including the description of the main system modules.

II. RELATED WORK
Research has already dealt with camera-based approaches for ergonomics assessment [9], [10]. However, most related approaches did not aim at an online ergonomics assessment. [11], [12], [13]. Approaches already proposed show the possibilities of optimizing ergonomics in human robot interaction, by adjusting robot movements based on pre-defined individual worker profiles [14] or real-time adjusting of robot movements to optimize worker's joint positions in welding operations [15]. Most of the work only focuses on postural assessment. In some approaches additional risk factors can be added manually by the user [9].
Technologies for motion analysis can generally be divided into optical and mechanical methods (Fig.1). Optical marker-based motion tracking is widely used and is characterized by high tracking accuracy. Advantages of markerless human motion capture are that The employee can freely perform movements and activities and is not restricted or affected by on-body sensors. This also eliminates the need to apply the optical markers and to calibrate the tracking system, which is a great advantage for regular and timeefficient use.

III. SYSTEM REQUIREMENTS
Considering the earlier discussed challenges resulting from the concept of the factories of the future, this paper proposes the following requirements to take into consideration for the system design. To be able to use the implement system in industrial environment, the sensor technology should meet the requirements of industrial use and at the same time be inexpensive to purchase. For marker-less estimation of the 3D human pose, the sensor system should be able to deliver the depth information and spatial (planar) information of the human joints. Table II gives an overview of minimum motion tracking requirements.

Latency <100ms
Latency will affect the real time capability of the tracking module and should be low.
Real-time 3D skeleton tracking, including human body joint tracking is proposed to be done via a middleware. For upper body postural assessment (posture classification and joint angle calculation) a minimum set of skeleton key points needs to be tracked by Tracking Module. For upper-body assessment this set of skeleton key point shall consist of minimum 9 joints: 1) Head, 2) Neck, 3) Torso, 4) Center pelvis, 5) Left shoulder, 6) Right shoulder. However, a whole-body postural assessment, including upper and lower body, is aimed at. The proposed system, shall support multiple camera sensors, allowing high flexibility in system design. Consequently, the user could use the Tracking Module and the whole system with multiple camera systems without having to adapt or change the algorithm.

IV. MODULAR SYSTEM ARCHITECTURE
The proposed system consists of five modules (following sections A. to E.), which can be seen as encapsulated system building blocks, see

A. Configuration Module
A central configuration module allows the system to be set up. User-specific profiles can be created and edited. Specific feedback time interval can be predefined here. Furthermore, this module allows connected databases and sensors to be selected, deselected, and set up.

B. Tracking Module
The tracking module provides hybrid detection of people, environment, and equipment, based on sensor and knowledge base data. Based on the detection and subsequent classification, the assignment of semantically and parametrically described human activities, operation specific action forces and types of workload -stored in the knowledge database -is carried out.

C. Ergonomics Evaluation Module
First, the methods for the ergonomic assessment of workrelated stress must be evaluated regarding their suitability for a partially automated assessment system. The most important requirement is that the ergonomic evaluation is to be achieved primarily through optically detectable data. The motion tracking should work without any on-body sensors, allowing freedom of action and movement of the user. A comprehensive ergonomic assessment is supposed to be based on main risk factors, see Tab. III. Furthermore, it should consider basic types of workloads (Table IV).   TABLE IV. POPULAR ERGONOMIC ASSESSMENT METHODS, INCLUDING TYPES OF WORKLOADS: L (LOAD HANDLING), PP (PUSHING ANDPULLING), P (POSTURE), R (REPETITIVE TASKS)

Method Including stresses caused by
Including stresses caused by For Ergonomic Assessment Module two main functionalities are proposed for the system design: (1) Online Posture Evaluation, (2) Long-term Ergonomics Evaluation.

1) Online Posture Evaluation:
Online feedback should primarily be based on posture analysis, as risk factors such as the number of repetitions and action forces are less suitable for consideration in an online feedback system. The influence of the latter factors is highly dependent on the temporal domain.We propose the implementation of postural assessment via using Rapid Upper Limb Assessment (RULA) [16], Ovako Working Posture Analysis System (OWAS) [17], "DGUV-Information 208-033, Appendix 3, Part 1" (DGUV 208-033 A3P1) [18] or Ergonomic Assessment WorkSheet (EAWS) section 1 [5].
OWAS aims at the posture evaluation of the whole body, while RULA is focused on the upper body. The guideline "DGUV-Information 208-033" of the German Social Accident Insurance (DGUV) recommends several assessment procedures for physical loads. DGUV 208-033 A3P1 categorizes angle ranges of various body joints according to a traffic light scheme. Considering the physiological range of motion, the angle categories are classified as neutral or acceptable ("green"), moderate or conditionally acceptable ("yellow") or final or unacceptable ("red"). By using this method for online assessment and user feedback, intuitive and user-friendly options for visualizing body parts are given.
2) Long-term Ergonomics Evaluation: Since most risk factors are assessed under the consideration of impact-time, it is indicated to consider these factors in a long-term assessment. As core for this module, we propose the use of EAWS [5]. Additional data, acquired via the short-term module can be included optional, too.

D. Worker Feedback Module
Multi-modal information-output possibilities via optical and acoustical feedback are foreseen. The focus will be on consideration of the basic principles of usability.

E. Robot Control Module
Several technologies could be found for designing an ergonomic workplace containing anthropometric adjustment of several properties. These technologies aim to reduce the physical workload. Some examples in automobile industry are 1. Adaption of working heights (by lifting the whole car) and gripping ranges.
2. Reduction of overhead assembly by turning the vehicle on its side for an easy access.
3. Use of special assembly seats to enable an easy entry to the interior of the car.
In general, planning of ergonomically, and worker individually optimized workplace is complex and cost expensive, as each station must be planned separately and worker specific anthropometrics and capabilities should be taken into account. In practice, the ergonomic-oriented workplace design is usually made considering the most relevant human percentile. According to these human properties (e.g. body height, body shape) and gender, the workspace could be designed in flexible way. However, the adaptation is usually performed manually by the human. In robot-based application, besides increasing the productivity, the collaborative robot could be considered as flexible assistance system which improves the human ergonomics. However, the robot system could lose its flexibility and enhanced priorities on the ergonomics due to the lack of programming-experience and health awareness by coworkers. One of the main benefits to have an onlineergonomics evaluation is that the results could be coupled with the robot control. Hence, the adaptation of robot parameters, which have no influence on the productivity or product-quantity, could be performed automatically. In other words, it closes the gap between the available functionalities by the robot and the health-requirements by the human. Furthermore, it can provide a complete individualized sociotechnical system.
According to individual anthropometrics, such as the size and body shape of the worker, we can draw conclusions about preferred working positions regarding working heights and gripping ranges -the robot will adapt his behaviour automatically. In addition to that, we are also able to adapt required forces for example for handling of components. Furthermore, it will be possible for the worker to adapt the assembly process individually. Given positions and forces are just suggestions by the robot system. Every worker should be able to change settings with very less effort. Due to this, we are going to reduce the planning issues to a minimum. In other words, the robot should be able to adjust independently and individually on the worker and the work task, without any re-programming. In contrast to known hybrid humanrobot solutions, this means that the task-oriented robot picks up the components and makes them available to workers in an ergonomic way (Fig. 3). The robot will be capable to identify individual ergonomic needs of the worker, as well as requirements of the assembly task.
By the end of a shift, workers frequently achieve several kilometres of walking distance and several trunk bendings, e.g. for picking up components out of containers or other storage equipment. This leads to lower productivity and more physical effort. The robot system will recognize the individual characteristics of the human to adapt the workplace individually -before the work process. This means that the robotic system will independently identify the characters of its partner (body size, body type, gender) to define and to adapt its relevant parameters (end-effector position, forces, pose, etc.) in ergonomic way for the human.
The robot provides an ergonomic assembly process with individual presets for e.g. working heights and forces.
 High number of DOF ensures that it can perform every assembly task.
 The robot can be used as a third hand for holding components, while the worker has both hands free and can exclusively focus on the assembly process.
 With respect to the exact installation position, robot and human can cooperate.
 It is possible to reduce assembly errors due to a defined positioning of the components.
All these issues aim to produce an ergonomic workplace adaptation based on recognizing of workers and then changing the parameterization of the assembly process regarding to it, such as handing-over pose, working heights, robot path, required forces and moments, etc. This ensures optimal working conditions for the workers during every shift.
Partially automated means, that physically demanding work is supported by assembly aids like balancer systems. While a component is held by the balancer system, the worker can move it easily without any physical effort. Such systems are only used for large and heavy components, like front ends, seats, and doors in automotive industry.
Nevertheless, positioning and clamping during the picking process remains manual. Especially picking up components out of a bin requires a certain physical effort. For example, bending over to reach the bottom of a container is hard work in the context of lifelong work and demographic change. Next to the physical demand, using balancer systems also requires sensitive skilled workers, because the assembly line runs along with the car. This means moving the car component with one hand within the speed of the assembly line, while assemble the component with the other hand. Beside those aspects manual positioning and clamping can cause damages on car components due to inappropriate positioning and jumpy movements of the worker because of high masses of the balancer and high moments of inertia correspondingly.
In addition, assembly aids are not or limited adaptable to a multitude of different product variants. In context of a future flexibilization of manufacturing, those assembly aids need to be flexible too. This will lead to overly complex assembly systems, which are difficult to use.

V. IMPLEMENTATION RESULTS
To achieve system modularity, system architecture has been implemented with Robot Operation System (ROS) on Linux Ubuntu 18.04 with ROS Melodic. Communication in ROS is accomplished by transferring information from one node to another node in a topic. Nodes publishing data are called publisher, while nodes requesting data are called subscriber. The first four modules have already been implemented as prototypes, namely A) Configuration Module, B) Tracking Module, C) Ergonomic Assessment Module and D) Worker Feedback Module (Fig. 2). Information is further processed in the second ROS node /ergonomic_assessment_module which is the realization of Ergonomic Assessment Module. Joint angle calculation is based on vector analysis. For certain joints, a calculation with pseudovector is required and has been implemented accordingly. The calculated joint values are stored in two arrays. The first array is used to for the short-term ergonomic assessment and its size is determined by the feedback time defined by the user. This array is reset after every feedback time interval. Second array stores all the joint values for the long-term ergonomic assessment until stopping the recording.
The third ROS node /visualization_module corresponds to Worker Feedback Module. The output flags for short-term ergonomic assessment of each joint are published via a custom ROS message /shortErgonomicAssessment.
The same procedure is used for the long-term assessment via the ROS topic /longErgonomicAssessment. The topic /longErgonomicAssessment is published when the system is closed (node is shutdown).
For system evaluation, Intel RealSense D435 was mounted 1.4 m high and aligned frontally to the user's face. After starting the system, the user inputs the required information such as user profile, feedback time, etc. This information is saved in the program routine and the tracking system is started simultaneously after that. With the tracking system, the human body joints positions are derived from the skeleton model. The environment for the test is standard industrial environment with temperature at 21°C (+/-0.3°C) and illuminance of 580 Lux at 1.5 m above factory floor ground.

A. Configuration Module
The Configuration Module is realized by using ROS services. The first service first_user_setting is called before modules B), C) and D) are executed. The first ROS service is currently used to set the worker's name and the feedback interval. The second service change_feedback_time is implemented to enable a change of the feedback time interval while the programme is running.

B. Tracking Module
In principle, the use of different sensor technologies is conceivable. Sensors that work based on Structured-Light, Active or Passive IR Stereo and Time-of-Flight technology may be used [20]. [21]. A comparison of potentially suitable and commercially available cameras was made for the purpose of decision-making (Table V). The main important aspect for choosing the middleware or SDK is that it can provide all the necessary body joints required for the ergonomics assessment and the method or calculation specification on which it is based. Furthermore, the requirements mentioned in section III are taken into consideration to choose the middleware. Common 3D skeleton tracking middleware are, openpose [22], alphapose [23], Nuitrack (Nuitrack™ SDK, 3DiVi Inc) and cubemos (cubemos Skeleton Tracking SDK, cubemos GmbH). Input data (RGB vs. RGB-D), output data (skeleton keypoints, data structure) and compatibility with several sensors are the main differences between the SDKs. 3D information and its tracking stability are main keys to achieve a steady system. Furthermore, light condition can be a vital key in such applications as it is affecting the result coming from RGB sensors. 3D data can be acquired from multiple camera views but may need time and cost expensive calibration processes. Thus, RGB-D sensors with Time of Light Active IR stereo technology are suggested to overcome the limitation of conventional RGB sensors. In the present system, human tracking is done with Nuitrack SDK based on a single Intel Realsense D435, allowing marker-less 3d human skeleton tracking. The camera is placed at a height of about 1.4 metres and is oriented perpendicular to the worker's torso to capture the worker's working posture. ROS-Architecture was implemented according to Fig. 4 and a screen is used as a human-machine interface. The extracted information of the human body joints is used as input for the calculation of the joint angles and these angles are used in subsequent steps for posture classification. Calculation of human body joint angles is done using analytical geometry and vector calculus in three-dimensional space. Depending on the key points generated by the specific SDK or algorithm, further keypoints are calculated if necessary, to assist the calculation of additional required human body joint angles for Ergonomic Assessment Module. The joint angle values are saved depending on an individually adjustable timer. The timer can be set via Configuration Module, which enables individual configuration of e.g. short-term and long-term ergonomic analysis, monitoring and reporting.

C. Ergonomics Evaluation Module
Postural assessment via using 1) DGUV-Information 208-033, Appendix 3, Part 1" (DGUV 208-033 A3P1) [18] and 2) Ergonomic Assessment Worksheet (EAWS) section 1 [5] was implemented. Assessment via DGUV 208-033 A3P1 is based on the categorized angular ranges of different body joints according to traffic light scheme. Considering the physiological range of motion, the traffic light categories are classified as neutral or acceptable ("green"), moderate or conditionally acceptable ("yellow") or extreme or unacceptable ("red"). The classification and evaluation by using a traffic light system complies with the standards EN 614, ISO 11228-3 and EN 1005-4 [24], [25], [26]. The colour code of the flag for the joint values assessment consists of three colours which, based on traffic light system (see Section IV). The detected upper body pose is printed as feedback on the visualization as well. As mentioned in Section V, long-term and short-term (online) ergonomic assessment were implemented. For short-term assessment a ROS topic can be flexibly subscribed. For long-term assessment and the recording of data, a tool to record messages in a ROS topic (ROS bag) is used.

VI. DISCUSSION AND CONCLUSION
The proposed system can be applied in several phases of the life cycle of a manufacturing system, particularly: (1) Development, (2) Construction, (3) Commissioning and (4) Operation. Already in the first phases, the system can be used for ergonomic assessment with the help of first physical mock-ups. During the operating phase, the investigations made in the planning phase can be validated. Even if changes to the work processes are necessary, the ergonomic parameters can be checked and adjusted quickly and costeffectively. The modular system architecture allows the system components to be adapted to specific application requirements.
First system prototype has already been implemented and tested in the test field, which is a fenceless robotic cell in industrial environment. Prototype implementation was done using Nuitrack SDK for 3D human skeleton tracking. Intel RealSense D435 (Active IR stereo) was chosen for first system prototype, since RealSense D435 represents an upper benchmark in its price range, considering resolution and performance.
First test series are currently being recorded and analysed to investigate the stability and accuracy of the human tracking. The capabilities of the current system prototype are limited due to using only one single depth camera (Intel RealSense D435, Intel Corporation). The used middleware for 3D skeleton tracking (Nuitrack SDK, 3DiVi Inc.) has the benefit of being compatible with different depth cameras but is not capable of multi-camera fusion and data processing.
Further efforts will be put into the design of the hardware agnostic system. For the future work, the system will be adapted to be compatible with multiple predefined skeleton models (SDKs), as the type of underlying skeleton model and accordingly tracked joints are depending on the middleware used for skeleton tracking. This will offer greater system flexibility and user friendliness. To allow maximum flexibility and interchangeability of the individual system components, a selective input switch will be implemented in future work to flexibly select the correct computational approach based on the appropriate skeleton model information provided. The current tracking module supports frontal tracking of the human up to a vertical tilt angle of ±35°.
In future work, a multi-camera system will be implemented to realize stable 360° human tracking. This will improve spatial tracking capability and robustness against object occlusion. The planned multi-camera 3D skeleton system will be benchmarked with several existing markerless human tracking approaches. Tracking accuracy will be analysed by using optical marker-based systems for ground truth.
Improved visualisation is planned. The presentation of the results of the ergonomics assessment is to be made more user-friendly. The use of an individually customisable dashboard system is envisaged. The user interface design will follow current UI/UX guidelines and acceptance. As the system is supposed to provide the recording, processing, and storage of personal data, it is necessary to ensure compliance with the requirements of the General Data Protection Regulation (EU) 2016/679 (GDPR), requirements in this respect will be considered in the further development.
The overall effectiveness of the methodological and system approach will be determined experimentally in the further development process. This implies that parallel to technical and scientific development, attention is also paid to applicability in the industrial environment. First discussions with potential users have confirmed the relevance of the system presented in this paper. New developments and results will be presented in future publications.