Usability of a mood assessment smartphone prototype based on humor appreciation

Humor appreciation is one of the determinants of individual's mood and can be assessed through jokes. We have developed a functional prototype called Humoris which asks users to select the funniest punchline and register their affective response to the jokes. Based on users' responses, the application predicts and displays their short-term mood using emoticons. Our smartphone prototype is evaluated using the 'think-aloud' method with 9 participants. Usability of Humoris was examined by System Usability Scale questionnaire which gave an average score of 79.44 (SD=8.08). Based on our findings, participants liked the application interface as well as the mood prediction but some of them found some jokes difficult to understand.


Introduction
Sense of humor is a human ability to process humor and appreciate the mental perspectives. A study [20] shows that decreased social interactions of individuals are related to dysfunction of affective perception and such deficiency can be seen in patients who suffer from major depression. Furthermore, Overholser [17] found a positive correlation between humor and lower depression, lower loneliness, and higher self-esteem. Sense of humor is believed to have a positive effect on stress reduction. Prior research by Yip and Martin [22] explored the positive role of humor in analyzing social dissect and interpersonal relationships. Also, two studies found a negative correlation between stress and sense of humor [13,11].  Humor is conceptualized in many different ways. One view explains humor as a cognitive experience which results in a mirthful state of mind and laughter as a way to display it [16]. Various techniques measure sense of humor including self-report humor scales and response questionnaires. However, these techniques are paper-based and don't necessarily measure sense of humor. For instance, Situational Humor Response Questionnaire (SHRQ) [15] is a 21-item scale that asks subjects to rate their response to pleasant and unpleasant situations. According to [19], it defines humor based on laughter frequency while laughter may happen without humor, therefore, it's not reliable. Nevertheless, there is a correlation between SHRQ and measures of personality and well-being.
Meyer [16] defines three theories for humor: relief, incongruity, and superiority. Incongruity theory proposes an anomaly to individuals in an unexpected or odd way. Uekermann et al. [21] link incongruity to resolution since humor processing involves incongruity detection and resolution to the body of a humorous situation such as a joke, cartoon, and comic. In addition, Goel et al. [10] suggest separation of cognitive and affective components of humor where cognitive component refers to the comprehension of stimuli and affective component points to the appreciation of the stimuli. Incongruity can be used in assessing both cognitive and affective components.
Joke and cartoon are popular means of humor assessment. Individual's joke telling assessment techniques and tool to measure tendency to initiate humor was developed by Bell et al. [5]. In this study, we preferred jokes since we couldn't hire a cartoonist to draw cartoons. In addition, our methodology is based on jokes.

Related Work
Several applications have been developed to track personalized mood [18,9,1,4,2]. MoA 2 developed by Bachmann et al. [2] is a context-aware smartphone application which provides the ambulatory assessment of mood, tiredness and stress level. The main features of this tool are (1) mood assessment and (2) mood recognition. This application is more effective in mood assessment compared to mood recognition which has an average recognition accuracy of 0.76.
Bozikas et al. [7] used Penn's Humor Appreciation Test (PHAT) to assess humor appreciation in schizophrenic patients. PHAT is a computerized test including 20 pairs of caption-less cartoons by Mordillo. Their findings showed impaired humor appreciation in schizophrenic patients. Boxikas et al. in another study [6] also used PHAT to assess the same impairment in patients with obsessive-compulsive disorder. However, their results were not statistically significant.
So far, no smartphone-based humor assessment tool has been developed. Furthermore, none of the current mood assessment applications have been developed to infer mood from humor appreciation level. The main objective of this study is to evaluate application user interface before deploying and testing mood prediction with patients who suffer from major depressive disorder.

Methodology
The application prototype presents a set of jokes in English to the users; users respond to the jokes and system predicts and reflects user's short-term mood ( Figure 3). Humoris is designed based on stage-based model of personal informatics systems [14]. A short description of the five stages is presented below: Preparation: Individuals interested in tracking their daily mood may choose Humoris as it uses humor appreciation criteria for mood prediction.
Collection: Affective score of each user is collected based on joke quiz results. Users can decide to receive a daily push notification or not.

Integration:
The application collects data and calculates short-term mood of the user automatically (system-driven).

Reflection:
Users can see their short-term mood represented by emoticons (Figure 1 (A, B or C)) after taking the joke quiz. They can also view their mood history.
Action: Based on the predicted mood, users receive a textual feedback. For instance, in case their mood is predicted as Figure 1. B, the application displays: Your mood is good today! Spread the smile!

Joke Quiz
In this study, we used five jokes in a quiz and each joke has four choices. It should be noted that all of the jokes are independent of individual's culture and race to avoid discrimination. Each of the four choices of a joke is supposed to complete the body of the joke but only one of the choices is the correct punchline, which involves incongruity element. The rest of the choices are logical, illogical, or irrelevant. One of the jokes is represented in Figure 2. To create each joke and the four choices we adopted a technique from a study by Uekermann et al. [21] . After selecting a choice for each joke, the users should tap on one of the emoticons as shown in Figure 3. In case users select the funny emoticon in 5 or 4 jokes, their short-term mood is above normal while 3 or 2 means normal, and 1 or 0 indicates below-normal mood. Figure 4 shows a normal mood prediction. Note that assigning mood level to the numerical joke scores has not been examined or validated before. However, we assume that mood prediction based on this technique leads to reliable results.
Three different colors are used to distinguish between the mood levels; green, blue, and orange show above normal, normal, and below normal mood, respectively. In mood history, a user can observe today's as well as previous mood assessment results. The distribution of the predicted mood levels are shown in the bottom of the screen in Figure 5.

Participants and Procedure
We recruited 9 participants (6 male, 3 female) from university campus to evaluate the usability of our application. All of the participants were student of average age 28.78 (SD=4.68). The distribution of the participants' education is 4 Ph.D. students, 1 post-doctorate, 3 graduate students, and 1 undergraduate student. The inclusion criteria was English language skill. None of our participants was native English speaker but they could speak English fluently for at least four years and their education program is in English.
We briefly introduced the application to each participant before beginning the experiment, during which we clarified the goals of our study. Then, the participants were asked to use and explore the application by themselves. Usability of Humoris was evaluated using 'think-aloud' method [12] followed by questions regarding usefulness, ease of use, and liked and disliked features of the application. We used iOS simulator for each participant to use Humoris application prototype. Each participant took the same joke quiz including 5 jokes. During the experiment, notes were taken and the entire procedure was recorded. Participants were asked to fill in the System Usability Scale (SUS) questionnaire [8] after using the application.

Implementation and Apparatus
The front-end of Humoris is implemented using React Native 0.52.0 and the experiment was conducted by Expo simulator on iPhone 7s plus.

Results
Each participant took approximately 7 minutes to work with Humoris. The results of our usability study are presented in the following sections.

Most liked features
In general, the design of Humoris was appreciated by all of the participants. All of them liked the application interface and colors used in the application including logo and data visualization. P1 and P9 particularly liked the colors used that reflect the mood in that particular day in the calendar (see Figure 5). P5 liked push notifications and disabling the feature as it is sometimes disturbing and said: I sometimes get confused by notifications sent from several applications and I'd like to turn them off.
P7 and P8 liked the simplicity of the user interface. P7 and P9 said that the system is user-friendly. P7 also encouraged the use of notification setting. P3 liked the circular progress bars used in Figure 4 and Figure 5.

Most disliked features
Some jokes were confusing for the participants. In addition, P4, P7, P6, and P8 were not sure whether to select the funniest punchline or the correct one. They suggested to change the title to "Which of the following can be the funniest punchline?". P1 and P6 did not like a specific joke because they didn't know the meaning of a word ('scented').

Suggestions
Some valuable suggestions were provided by the participants to improve the system. For example, P3 suggested that it would be easier for a user to read and respond to the jokes within the notification without having to open the application. P3 also suggested customizing the jokes based on users' age range. P4 suggested providing practical feedbacks such as useful activities on how to improve a low mood. P5 mentioned that the application feedback on predicted short-term mood may negatively affect users who suffer from depression.

SUS score
Each participant filled in the SUS questionnaire. The average score of 9 participants was 79.44 (SD=8.08).

Discussion and Future Work
In this study, we assessed Humoris interface and we received several useful feedback to improve the application. Although our methodology for joke quiz is based on patients who suffer from mental disorder, we have not tested our application with these patients yet. It has been planned to conduct a clinical trial with clinically-depressed patients and validate the results. One of the issues with our methodology is that mood level is predicted based on total scores out of POSTERS MobileHCI'18, September 3-6, Barcelona, Spain selecting an emoticon for each joke but there is no method to validate the thresholds. For instance, we cannot assert that score 3 indicates normal mood level before collecting user data. Moreover, none of our participants was a native English speaker which made it difficult for them to comprehend some vocabularies such as 'scented'. In future works, jokes should be provided in native language of the users to obviate language barrier.
Bangor et al. [3] assigned seven adjective ratings to each mean SUS score ratings. According to their classification, Humoris user interface is ranked as excellent since the SUS score obtained from our study is above 71.4 (SD=11.6) and below 85.5 (SD=10.4). High scores in ease of use, simplicity, consistency, and functionality indicate that Humoris has a user-friendly interface. However, according to SUS results, almost half of our participants may not frequently use Humoris. One method to improve future usage of Humoris is to use other means of humor assessment like cartoons. Using cartoons instead of jokes could be more useful but not feasible at the moment due to expenses of hiring a cartoonist for drawing a number of cartoons. During the usability experiment, we observed that the users who couldn't get the correct punchline were reluctant to respond that the joke was funny although they got the joke after seeing the correct choice. Some of the participants became disappointed as they noticed they had selected an incorrect choice. Consequently, we cannot ignore the negative impact of wrong choice selection on affective response. However, the mentioned behavior should be studied in a separate work.
We used both cognitive and affective components of humor.
Although we didn't use cognitive score for personalized mood prediction, it can facilitate creating an adaptive application. To clarify, suppose that a user doesn't understand a joke and the punchline is not clear for him/her. An option to capture the confusing response could be used to propose an easier joke by considering cognitive scores. This technique obviously requires the degree of difficulty for each joke. Therefore, users should rate each joke and the average score would be reported for each of the jokes in another survey.
We examined the use of humor appreciation through emoticons on mood prediction. The positive response to the main goal of Humoris is promising although other means of humor assessment including cartoons can be taken into consideration in future development. Also, we would like to develop a research tool that evaluates the short-term mood of the users by having them take a joke quiz in the morning and asking each of the users about the impact of taking a joke quiz on their mood during the day. By comparing predicted mood and subjective mood assessment by users, we can calculate the accuracy of mood prediction and evaluate the positive, negative, or neutral impact of the jokes.

Conclusion
Humoris is a smartphone prototype designed to predict short-term mood of the individuals. Emoticons are an engaging way of capturing humor appreciation. By combining emoticons with jokes to assess affective response, short-term mood of individuals can be estimated. Also, mood history can give useful feedback to the users. Participants liked the user interface and design of Humoris although some of the jokes used in the application were not easy to understand which led to the unlikely future usage of the application by nearly half of the participants.