Dynamic Displays at Wrist for Real Time Visualization of Affective Data

Emotions are vital to our lives but could be difficult to recognize and understand. Traditional visualizations of emotions tend to be time-series graph on screen displays limiting user engagement in their real-time sense-making. This paper explores the feasibility of smart materials for developing novel dynamic displays on skin for real time visualization of affective data. We report prototyping two such displays and their evaluation with 6 participants, and discuss their qualities such as ambiguity, slowly unfolding change, and lack of light emission together with their temporal constraints and private-public tension for affective meaning disclosure.


Introduction
Emotions are vital part of our everyday lives signaling events of significance, but sometimes difficult to recognize and understand. The emergence of wearable technologies for tracking physical and mental health [12] shows increased interest in self-understanding and behavior change. Traditional wrist-worn displays represent affective data in time-series graphs [8] which provide limited user engagement for reflection. This paper explores the feasibility of smart materials for developing novel, thin and flexible wrist-worn displays for real time visualization of affective data to better motivate reflection [10]. We report the design and evaluation of two prototypes exploiting the value of ambiguous representations of emotional intensity.

Fabrication of on-Skin Output Displays
Our focus on wrist-worn displays guided us to HCI work on thin, easy to fabricate, and flexible on-skin thermochromic displays [15]. Thermochromic displays undergo color change when heat is applied; they are non-emissive but sensitive to temperature changes in the form of paper-like displays [7] or dynamic textiles [1]. An example is DuoSkin [5] using adhesive tattoo paper to fabricate displays resembling body art. Thermochromic displays have low power requirements [7], slow response time, and subtle qualities, but low resolution because precise heat control is challenging.

Emotion Visualization
Much HCI research has focused on designing systems for emotion recall [11], processing [14], or visualization. For instance, AffectAura [8] presents a lifelogging tool to visualize valence and arousal in the form of a timeline based interactive interface. Other approaches providing time-series graph-based visualizations include tools for self-tracking moods [19]; while easy to understand, they provide limited support for reflection on real time data [6,10]. In contrast, other systems have explored ambiguous clothing based-displays [1] to represent emotions in social settings [4], which support stronger user engagement and reflection. For example, AffectiveDiary [6] displays historic affective data into colorful and abstract body shapes, while Affective Health [10] uses electrodermal activity, electrocardiogram, and accelerometer data to create ambiguous designs [2] aimed to engage users in reflection on their affective states. To conclude, emotion visualization displays support reflection but are limited because their mobileor desktop-based interfaces are not always on sight. In contrast, on skin-output displays potentially always on sight, but seldom integrate biosensors. Our work uniquely integrates these two strands of work.

Prototyping Affective Displays
To support real time reflection on affective data and engagement within this practice [13], we developed two color changing displays for ambiguous representations of arousal: Spiral ( Figure 1) and Heart ( Figure 2) differing in color, shape and movement. We employed a material exploration method [16] consisting of an experimental and playful approach for fabricating three layers [5,18]: thin and low-cost thermochromic materials layer, custom shaped heating mechanism layer, and insulation layer ( Figure 3). For material layer, we explored thermochromic liquid crystal sheet and pigments for their ability to display color changing effects and discard the ones actuating above 40°C for safety reasons. The choice of heating element is crucial as it can limit the resolution of thermochromic displays [15]. We explored copper, aluminum and nichrome as heating elements and chose nichrome wire over others because of its relatively high resistance. Insulation is important for thermal applications to avoid skin burn, and we considered several materials such as epoxy resins, polypropylene and polyimide insulation tape. The whole assembly is placed on polypropylene substrate for insulation. Increase in arousal is conveyed by varying current in nichrome wire, which generates color changing from red (left), green (middle) to blue (right) in a spiral shape from inside out.

Figure 2:
The Heart display involves a digitally created art design printed on adhesive tattoo paper. The design was transferred to paper and a 30-35 °C thermochromic pigment was applied at the center. A heart shaped nichrome wire was placed on polypropylene substrate. The thermochromic and heating layers were integrated with clear tattoo paper. Increase in arousal is represented by the change in heart's color from purple (left) to pink (right).

User Study
The study explores people's understanding of thermochromic displays visualizing ambiguous affective data. We recruited 6 students (3 males and 3 females, age ranged 24-36), and used autobiographical recall for inducing emotions [3], i.e. writing about memories of anger/fear, and happiness/excitement, while wearing the prototypes on their wrist. We asked about their perception of displays, interpretation of affective data, and how they envision using the affective displays. Sessions were audio recorded and transcribed for content analysis, and participants were rewarded £10.

Findings
An important finding is that as opposed to screen displays, thermochromic materials are more preferred.

Discussion
We designed and evaluated two dynamic displays visualizing emotional arousal. Study findings suggest their specific qualities differing from traditional screenand time-series graph-based visualizations of emotions. Participants liked them more, and appreciated the slowness of real-time changes in arousal, their more cloth-like interface, as well as the inherent ambiguity of affective data whose control allows navigating the trade-off of private-public disclosure of meaning.

Temporal Constraints of Dynamic Affective Displays
Findings also indicate that while a slow change of colors is appreciated, too slow changes become a hindrance. In this respect, the distinct material properties of thermochromic liquid crystal sheet and pigment offer different affordances. The former is activated quicker (6 vs 15 seconds) when arousal increases and is less inertial in reversing to original color (2 vs 75 seconds). The temporality of affective display also relates to longterm use. While initial meaning tends to be shaped by cultural perceptions of colors, long term use may allow people to test and revise these meanings.

Taming the Ambiguity of Affective Displays
The two affective displays differ in colors, shapes, and movements allowing the comparison of people's understanding. In contrast to Heart display, the multiple colors of Spiral display support stronger meaning making for mapping discrete emotions to different colors. With only two colors available and limited intermediate stages of color changing, Heart display offers a binary visualization of emotions. In addition to multiple colors, the Spiral design is also appreciated as the circle's diameter provides additional information. Our findings extend Gaver's ambiguity [2], by reinterpreting the relationship between artifact, wearer and public as suggested for clothing-based displays [1]. We argue however, that the coupling between artifact and wearer is even tighter for affective displays representing private bodily data.

Tension of Private-Public Affective Meaning Disclosure
All participants envisaged using affective displays for self-or co-regulation of emotions with loved ones. They showed openness to share publicly their positive emotions and privately the negative ones. While participants appreciated the inherent ambiguity of colors' meaning, they liked even more the ability to control it through personalization of colors. To summarize, our findings extend previous work on clothing-based thermochromic displays [1,17] and skin conductance in social interactions [4]. They suggest the value of added ambiguity [2] regarding color and speed of change of affective content.

Conclusion
We developed two dynamic affective displays and evaluated them with 6 participants. Findings indicate that participants were able to construe affective meaning and to envision contexts of use. We discussed the qualities of affective displays in terms of ambiguity, slowly unfolding change, temporal constraints and private-public disclosure of affective meaning.

ACKNOWLEDGMENT
This work has been supported by AffecTech: Personal Technologies for Affective Health, by the H2020 Marie Skłodowska-Curie GA No 722022.