Mid-Air Haptic Interfaces for Interactive Digital Signage and Kiosks

Digital signage systems are transitioning from static displays to rich, dynamic interactive experiences while new enabling technologies that support these interactions are also evolving. For instance, advances in computer vision and face, gaze, facial expression, body and hand-gesture recognition have enabled new ways of distal interactivity with digital content. Such possibilities are only just being adopted by advertisers and retailers, yet, they face important challenges e.g. the lack of a commonly accepted gesture, facial expressions or call-to-action set. Another common issue here is the absence of active tactile stimuli. Mid-air haptic interfaces can help alleviate these problems and aid in defining a gesture set, informing users about their interaction via haptic feedback loops, and enhancing the overall user experience. This workshop aims to examine the possibilities opened up by these technologies and discuss opportunities in designing the next generation of interactive digital signage kiosks.

For instance, advances in computer vision and face, gaze, facial expression, body and hand-gesture recognition have enabled new ways of distal interactivity with digital content. Such possibilities are only just being adopted by advertisers and retailers, yet, they face important challenges e.g. the lack of a commonly accepted gesture, facial expressions or call-to-action set. Another common issue here is the absence of active tactile stimuli. Mid-air haptic interfaces can help alleviate these problems and aid in defining a gesture set, informing users about their interaction via haptic feedback loops, and enhancing the overall user experience. This workshop aims to examine the possibilities opened up by these technologies and discuss opportunities in designing the next generation of interactive digital signage kiosks.

BACKGROUND
Digital signage and pervasive displays have become virtually ubiquitous due to falling display hardware costs and a rising demand for location-based advertising. Such systems are now commonplace in shopping centers, airports and public spaces and form the main medium for advertising and broadcast announcements. Recent developments in face, body and hand-gesture sensing are now enabling an entirely new class of display applications that can cater and tailor their content to specific audience or customer needs, see Fig. 1a. Coupled with connectivity and personalised data for ad-targeting, signage systems are beginning to transition from simple broadcast systems to data-rich interactive platforms [1]. Similar trends and opportunities to digital signage are observed with interactive kiosks. Namely, low hardware costs and high staffing costs have resulted in an increase in public information and transaction kiosks [2]. With the recent software advancements in body scanners and augmented reality graphics, interactive kiosks have expanded to include for example virtual dressing rooms that enable shoppers to virtually try on clothes of different size, fit or style, see Fig. 1b. These systems need to provide an efficient and pleasant service, be accessible to a wide range of users and be usable with minimum prior instruction.    As signage and kiosk systems are transitioning from static displays to rich, dynamic interactive experiences, the new technologies supporting them are also evolving. Moreover, and unlike touchless screens, touch-screens have many issues when it comes to interactive digital signage and kiosk applications: hygiene and cleaning requirements, robustness against extended use and potential damage, securing access to the display control panel, responsiveness, and finally reachability requirements that compromise viewability and location [1]. Therefore, there is a few good reasons for interactive digital signage and kiosks to move away from touchscreens and enable users to interact with machines and content from a distance, without the need for any physical contact with the controller itself Touchless interfaces have been extensively deployed in the context of AR/VR: Microsoft's Hololens allows the user to control AR applications using hand tracking and gestures. The Leap Motion Controller concentrated its efforts on providing software tools to enable hand tracking in conjunction with VR headsets. Recent progress in Computer Vision allows for precise recognition of hand and body landmarks from basic monocular or stereo cameras [3,4]. This in turn is expected to democratize future applications of hand tracking and gesture recognition as well as the incorporation of body language into the interaction loop. Further advances in facial expression and gaze recognition ( Fig.  1c.), as well as detection of emotion from environmental sensing [5,6], expand the set of HCI tools to be explored in signages applications. The immediateness provided by touchless controllers comes with a significant draw-back: the lack of tactile feedback coming from the interaction with a physical device. Further, while emotion recognition is already widely used in marketing research [7] the combined array of affective computing and affective haptics [8] are still relatively rare in real world applications partly due to the lack of an easy, effective, and practical way to elicit, enhance, or influence the emotional state of a human by means of sense of touch. In touchless physicality can result in an 'affective disconnect' [9]. The latest developments in ultrasonic mid-air haptic technology, see Fig. 2, have made it possible to build interactive touchless interfaces that also incorporate real-time tactile feedback [10,11]. Recent studies suggest that mid-air haptic feedback can affect the emotional state of users and allows to significantly increase the engagement of interaction [12]. Thus, applied to interactive digital signage and kiosks, ultrasonic mid-air haptics offers users a multi-sensory experience that can potentially dramatically improve the overall experience, increasing brand engagement, dwell time and brand recall [13]. In the case of interactive kiosks and virtual dressing rooms, mid-air haptic feedback may also be able to instill physical properties to virtual objects such as the sensation of textured roughness [14] and improve touchless interaction performance [15].

WORKSHOP GOALS
This one-day workshop will feature demos, 3-minute presentations, several mini-keynotes, group discussion sessions, and challenge and response group breakout sessions during which we will collectively endeavor to identify challenges and opportunities, define new use case scenarios, suggest new design guidelines, and formulate and initiate collaborative research studies for participants to embark on post-workshop. The workshop's main goal therefore is to bring together the growing mid-air haptic research community and industry experts (see CHI2016 and CHI2018 workshops [17,18]) to advance the challenges and opportunities related to the design of next generation mid-air haptic interfaces for interactive digital signage and kiosks. Namely, we are interested in at least these questions: -How can we overcome "display blindness" and what new spatial interaction zone models can be constructed through the addition of mid-air haptics? -Call to action: how to best evoke passerby interest in the interactive systems and later maintain engagement? -Is there a set of specific gesture or body interactions that are better suited for specific scenarios? -What emotions can we induce with haptic feedback and how to measure related effects? -How can we cater to multi-user (collaborative) interactions? Can we improve accessibility? -What design guidelines should user experience designers adhere to?

KEY CHALLENGES
The workshop will focus on the following identified key themes and challenges related to the next generation mid-air haptic interfaces for interactive digital signage and kiosks: 1) Interactivity call to action, 2) New use cases (including multi-user and accessibility), 3) Interaction techniques, and 4) Design guidelines. The workshop will be organized into two group discussion sessions covering these 4 key challenges. The organizers will adapt the key challenges according to the submissions to the workshop (this process is described in the following sections). In view of these challenges, the workshop aims to: 1. Connect and enhance synergies between a growing community of touch-less haptic researchers, application and product designers.

2.
Facilitate an open discussion between academia and industry on the challenges they face when creating digital content for the next generation of mid-air haptic interfaces for digital signage and kiosks.
3. Brainstorm and design concept prototypes. 4. Define new research challenges in the field of digital signage which are enabled by mid-air haptics.

ORGANIZERS
The workshop organizers have extensive experience in organizing and participating in many similarly themed workshops, including two successful CHI 2016 and 2018 workshops that attracted 25 participants [17,18] and resulted in new collaborative publications and research grant funding. Being an industry/academic -led workshop, the organizers are in a unique position to offer the HCI community unique insights on this growing field. For example, these may include market trends on the use of related enabling technologies such as affective computing and computer vision, and also discuss customer pain-points. The organizers bring together expertise in retail solutions from Alibaba Mark Perry leads the HCI group at Brunel University. He and has worked on the design and evaluation of public and situated displays over the last 15 years, with a focus on understanding social interaction around them. Jörg Müller is Professor of Computer Science at the University of Bayreuth and is interested in modelling, simulation and optimization of human-computer interaction using dynamical systems models, ultrasonic levitation interfaces, and augmented and virtual reality. Myroslav Bachynskyi is a postdoc researcher at Computer Science Department at Bayreuth University. He has several publications on human biomechanics, mid-air movement modeling and ultrasonic levitation at many international HCI conferences and journals

WEBSITE
The workshop call for participation will be available at: www.ultrahaptics.com/news/events/chi-2019/ An online registration form (e.g. via Google Forms) will allow interested researchers to submit their expressions of interest. Once finalized, the program of the workshop will be posted online. A postworkshop section will be available on the website containing extra material, presentations from the attendees, pictures, videos, and notes from the workshop sessions.
6 PRE-WORKSHOP PLANS 6.1. Workshop advertisement: Our call for participation will be available on the workshop's website and distributed to the haptics and HCI community via forums, mailing lists, and social media. The organizers have excellent connections in both academia and industry and will make sure that key people in relevant communities (HCI, haptics, digital signage, advertising and interactive kiosks) will be made aware of the workshop and inform their network of peers. Finally, we aim to be a broad church and will welcome researchers from both academic and industrial backgrounds as well as practitioners of varying juniority and seniority, interests and expertise. 6.2. Workshop expressions of interest format: Participation will require the submission of a short expression of interest (approximately 2-4 pages) in which applicants will highlight their research in relation to the workshop topic. We appreciate that since this is a very fast growing and cross-cutting space some participants will not have sufficient track record and therefore we will encourage the submission of position and concept papers. The theme of these papers could be a technological issue, design-oriented challenges or a personal or corporate vision of how the emerging field of mid-air haptics should evolve in order to cater for the next generation of digital signage and interactive kiosks. Themes should be linked to the applicant's personal research or product design experience. 6.3. Applications review and selection: These expressions of interest will be reviewed to assess their relevance to the workshop topics: If we receive more than 25 applications, we will select those that have the best potential to positively foster diverse discussions. The organizers will also ensure that new comers to the filed, such as less experienced researchers or students, will have a chance to be selected as participants and present their papers. 6.4. Definition of the workshop program: After the selection process, the organizers will review the common themes that transpire through the expression of interest paper submissions and use these themes to refine the key challenges and reflect these in the group discussion topics and challenge group breakout sessions. These will define the workshop's structure. In each of these sessions, a moderator (one of the organizers) will actively interact with the audience to stimulate discussion around the topic and outline the key points that will be useful for the whole group debate. The list of discussion topics, workshop program and selected expressions of interest will be made available via the workshop website. 6.5. Workshop recording and documentation: Finally, the organizers will arrange for the recording and documentation of the workshop. Much of this material will be used for disseminating on social media but will also form the basis for future publications and foster new collaborations between participates. This approach has been successful in previous workshops organized by the authors on related topics [17,18].

WORKSHOP STRUCTURE
The agenda of this one-day workshop will be based on the themes that emerge during the selection process and is aimed to merge presentation with the interactive collaboration, group discussions and a live demo session. The workshop will start by introducing the submitted papers, identified themes and common topics. The participants will then have dedicated slots to showcase their contribution in a short 3-minute madness presentation format that is interleaved with several mini-keynote presentations from industry and academia that will focus on selected topics. These may be accompanied by short video clips or demos. Interactive group discussion and challenge sessions will form the main part of the workshop and are the ones that will ultimately contribute towards the workshop goals and address the key challenges above-mentioned. Note that the discussion sessions will also be alternated with coffee and lunch breaks during which participants will present their work to each other and interact with demos. The change of pace will keep everyone focused and engaged. Demos, videos and screenshot presentations of participant's work will provide concrete examples encouraging discussions. The organizers will lead the two discussion sessions and encourage participation from all attendees. Next, the participants will form a few small groups, and will engage in a Challenge/Response session where they will be presented with design challenges for which they have to think of an appropriate solution and present their response to the whole workshop.

POST-WORKSHOP PLANS
The outcome of each of the brainstorming and group sessions will be made available to participants on the workshop website and will be in the form of videos and organized notes, highlighting the research challenges and design ideas that emerged from the discussions. A professional sketch artist will be hired to review the challenge responses and produce high quality info-graphics, storyboards and diagrams to exemplify the challenge responses, discussion topics, interaction techniques and use cases. These notes will later be made available online (see Fig. 3).

ACKNOWLEDGMENTS
This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 801378.

Mid-Air Haptic Interfaces for Interactive Digital Signage and Kiosks
Digital signage systems are transitioning from static displays to rich interactive experiences with the aid of new enabling technologies. Advances in computer vision have enabled face, gaze, facial expression, body and hand-gesture recognition thus enabling new ways of interactivity with digital content. A few challenges faced by advertisers and retailers is the lack of a commonly accepted gesture, facial expressions or call-to action set. Moreover, public displays that use touchscreen technology face issues with hygiene and cleaning requirements, robustness against extended use and potential damage, securing access to the display control panel, responsiveness, and finally reachability requirements that compromise viewability and location.
Of particular interest to this sector and to this workshop are mid-air haptic interfaces that couple computer vision and affective computing with haptic feedback through the use of focused ultrasound. Mid-air haptic interfaces have been shown to aid in defining a gesture set, affect the emotional state of users, significantly increases user engagement with the interactive content, increase brand engagement, dwell time and brand recall while also enhancing the overall user experience.
The goal of this industry/academic co-led one-day workshop is to bring together a group of experts in disciplines from haptic research, affective computing, pervasive displays and multi-modal interaction, in order to foster discussion about the challenges in the design and implementation associated with next generation of mid-air haptic interfaces for digital signage and kiosks. Through brainstorming and break-out sessions, we will design concept prototypes and define new research challenges in the field of digital signage which are enabled by mid-air haptics and affective computing. We invite potential participants to submit a 2-4 page abstract in the format of an ACM Extended Abstract showing how the applicant's work fits into the topics of the workshop and should include a theme that the applicant feels is important to the field and would generate valuable discussions during the workshop. Since this is a relatively new area of research, we welcome submissions of position papers that propose new ideas and challenges. In case of a multi-authored abstract, it should be made clear which authors will be attending the workshop. For each accepted paper, at least one author will have to register to both the workshop and at least one day of the conference.