Touch the Story: An immersive mid-air haptic experience

Haptic devices have often been used to enhance a variety of audio experiences such as listening to music, meditating, wayfinding, accessibility, and communicating. In most cases, the haptic interface is wearable or handheld and therefore suffers from limitations related to ergonomics or a limited palette of haptic sensation effects. In this paper, we present a touchless audio-haptic demonstrator experience that enhances the immersive narrative of an emotional short story. To do so, we have created an audio-haptic mapping that is semantically congruent and have synchronized the presentation of audio and haptic effects to the narrative timeline. The haptic effects presented to the user’s palm are both spatially and temporally modulated so that they convey a rich palette of sensations (e.g., tapping, direction, rotation, rain, electricity, etc.) that are triggered by keywords or events in the story.