Classifying Excavator Collisions Based on Users' Visual Perception in the Mixed Reality Environment

Visual perception plays an important role for recognizing possible hazards. In the context of heavy machinery, relevant visual information can be obtained from the machine's surrounding and from the human-machine interface that exists inside the cabin. In this paper, we propose a method that classifies the occurring collisions by combining the data collected by the eye tracker and the automatic logging mechanism in the mixed reality simulation. Thirteen participants were asked to complete a test scenario in the mixed reality simulation, while wearing an eye tracker. The results demonstrate that we could classify the occurring collisions based on two visual perception conditions: (1) whether the colliding objects were visible from the participants' field of view and (2) whether the participants have seen the information presented on the human-machine interface before the collisions occurred. This approach enabled us to interpret the occurring collisions differently, compared to the traditional approach that uses the total number of collisions as the representation of participants' performance.