Seeing Is Believing but Feeling Is the Truth: Visualising Mid-Air Haptics in Oil Baths and Lightboxes

Ultrasound is beyond the range of human hearing and tactile perception. In the past few years, several modulation techniques have been invented to overcome this and evoke perceptible tactile sensations of shapes and textures that can be felt, but not seen. Therefore, mid-air haptic technology has found use in several human computer interaction applications and is the focus of multiple research efforts. Visualising the induced acoustic pressure field can help understand and optimise how different modulation techniques translate into tactile sensations. Here, rather than using acoustic simulation tools to do that, we exploit the micro-displacement of a thin layer of oil to visualize the impinging acoustic pressure field outputted from an ultrasonic phased array device. Our demo uses a light source to illuminate the oil displacement and project it onto a screen to produce an interactive lightbox display. Interaction is facilitated via optical hand-tracking technology thus enabling an instantaneous and aesthetically pleasing visualisation of mid-air haptics.

tactile sensations. Here, rather than using acoustic simulation tools to do that, we exploit the micro-displacement of a thin layer of oil to visualize the impinging acoustic pressure field outputted from an ultrasonic phased array device. Our demo uses a light source to illuminate the oil displacement and project it onto a screen to produce an interactive lightbox display. Interaction is facilitated via optical hand-tracking technology thus enabling an instantaneous and aesthetically pleasing visualisation of mid-air haptics.

INTRODUCTION
Ultrasound manipulation using a phased array of speakers is growing in popularity in the human computer interaction community for haptic and multimodal applications [1]. Therefore, understanding the behaviour and tactile perception of focused ultrasound in space and time has become an important area of research. Several methods for visualising focused ultrasound exist, however they tend to require either digital help in the form of computer simulations [3] [2], or expensive equipment such as a laser Doppler vibrometer [5]. Cheaper and more accessible methods include the use of dryice or Schlieren photography [4]. However, none of these techniques offers a one-to-one interactive visualisation of the haptics being projected by the ultrasound phased array. Our demo achieves that by exemplifying the famous 17th century idiom by English clergyman, Thomas Fuller, "Seeing is believing, but feeling is the truth", where everyone can see the haptic projection on the lightbox in Figure.1, but only the user can truly feel it.

DEMO SETUP AND INTERACTION
Our demonstration is composed of a laptop, an external Ultrahaptics Stratos Explore (USX) development kit, and a lightbox containing an oil bath and a second USX device. The laptop is located next to the lightbox and is used to control the two USX devices through a software application we developed (see Figure.1).
The lightbox itself has five main components: a solid frame made by aluminium beams; an oil bath built of clear acrylic sheets and some sesame oil; a USX device pointing down towards the oil bath; a powerful point light source incident at 50 degrees to the oil bath; tracing paper screen that captures the oil displacement shadow reflections of the light source. To improve the resulting shadow effect, and reduce possible contamination of the oil, we shroud the frame of the oil bath in black cloth.
Using this setup, ultrasonic mid-air patterns are directed down towards the thin layer of oil by the internal USX device. The pressure produced by the focused ultrasound corresponding to different mid-air haptic patterns pushes a localised force onto the oil inducing a localised change in thickness. The contrast between normal and thinner thickness is not apparent to the naked eye, which is why we shine a high intensity light source onto the bath, at an incidence angle that is smaller than the critical angle for total internal reflection, causing it to refract and reflect onto an adjacent tracing paper screen. Due to the difference in oil thickness light will refract and reflect differently thus revealing the mid-air haptic pattern on the tracing paper screen.
All the haptics produced for the oil bath are a replication of the ones also displayed by the outer USX that users can directly feel and interact with by placing their hand approximately 20 cm above it. The hand is recognized and tracked using a Leap Motion controller. A demo application was built within Unity3D using C# that projects haptic sensations through the USX onto the user's hand. The user can modify the parameters of the sensation through the application, enabling or disabling features such as hand tracking or adjusting the haptic output such as shape, size, draw frequency and modulation technique. A user can therefore see on the lightbox screen, in real-time, what they are feeling through the outer USX. Moreover, the demo allows users to experience how different interactions and effects impact the output of ultrasound haptics.

CONCLUSIONS
The proposed interactive multimodal system can be easily assembled using off the shelf components and can be effectively used for multiple different scenarios. Namely, it can be used for haptic debugging to quickly see and identify problems with the USX output that cannot be felt. Further, the system can be used for the easy visualisation of haptic patterns allowing for greater understanding and intuitive explanations of what each haptic is drawing and how focused ultrasound haptics works. Finally, the demo is a fun and aesthetically pleasing experience.