TooltY: An approach for the combination of motion capture and 3D reconstruction to present tool usage in 3D environments

Visualization techniques for usage of tools, handicrafts and assembly operations are employed for demonstrating processes (e.g. assembly instruc-tions). Until today, most commonly used techniques include written infor-mation, sketches in manuals, video instructions, etc. The evolution of technolo-gy nowadays has generated mature methods for transforming movement to digital information that can be processed and replicated. Motion capture together with 3D reconstruction techniques can provide new ways of digitizing handi-crafts. At the same time, Virtual Humans can be used to present craft processes, as well as to demonstrate the usage of tools. For this, the tools utilized in these processes need to be transformed to the digital world. In this paper, we present TooltY, a 3D authoring platform for tool usage presentation in 3D environ-ments, in order to demonstrate simple operations (e.g. usage of hammer, scis-sors, screwdriver), where the tools are the product of 3D reconstruction. The movement of the Virtual Humans derives from motion capture, while for the one of the tools, a novel approach is used, for inducing the tool motion from the human motion capture. The products of TooltY are Virtual Environments that can be experienced in 3D or through immersion in Virtual Reality.