SiSiMo: Towards Simulator Sickness Modeling for 360° Videos Viewed with an HMD

Users may experience symptoms of simulator sickness while watching $360^{\circ}/$VR videos with Head-Mounted Displays (HMDs). At present, practically no solution exists that can efficiently eradicate the symptoms of simulator sickness from virtual environments. Therefore, in the absence of a solution, it is required to at least quantify the amount of sickness. In this paper, we present initial work on our Simulator Sickness Model SiSiMo including a first component to predict simulator sickness scores over time. Using linear regression of short term scores already shows promising performance for predicting the scores collected from a number of user tests.


INTRODUCTION
While enabling immersive experiences, 360 • /Virtual Reality (VR) videos may lead to undesirable effects on the user's well-being referred to as cybersickness or simulator sickness. Predicting simulator sickness while watching 360 • videos is of relevance to ensure viewing comfort and safety. The symptoms of simulator sickness may not be eliminated completely by making advancements in software and hardware (such as tracking, resolution of the device and displayed video, higher refresh rate, improved production quality with regard to e.g. stitching, etc.). If the omnidirectional video display via HMD is optimal, then simulator sickness will become motion sickness [4]. In [2], Kim et. al. proposed a VR sickness assessment CNN (VRSA) that takes into account motion patterns of VR videos and predicts sickness score for a test video. In [3], Kim et. al. proposed a framework called "VR sickness predictor" (VRSP) that considers perceptual motion and statistical content features to predict sickness scores on an individual subject level and per video. In [5], Padmanaban et. al. designed a sickness predictor that estimates the nauseogenicity (i.e. tendency to introduce nausea) of 360 • stereoscopic videos.
All the aforementioned studies predict sickness for specific videos or VR scenes, or at an individual-subject level. None of the reviewed studies addresses sickness more holistically with respect to its evolution over time and in light of possible pauses or other actions during a viewing session.
The contributions of the paper are as follows: (1) First general concept of simulator sickness model applicable to complete viewing sessions, (2) prediction of simulator sickness over time, taking breaks into account.

SISIMO CONCEPT
In our running work on developing SiSiMo, we assume that simulator sickness results from a temporal succession of different phases * e-mail:alexander.raake@tu-ilmenau.de that may positively or negatively impact simulator sickness and result in a certain level of overall comfort. Each phase is addressed by a specific module within SiSiMo (cf. Fig. 1). Here, we distinguish phases of viewing (module M V , may include audio), where e.g. highmotion video may lead to an increase of simulator sickness, pausing (M P ), assumed to lead to a decrease in simulator sickness, and rating (M R ), reflecting that respective research typically involves ratings of simulator sickness or additional aspects such as video quality. It is noted that the latter will also include aspects of rating bias that depend on the actual rating procedure used. For the viewing module M V , approaches as discussed in Sec. are being considered. In addition, the model concept includes inter-and intra-phase adaptation ("A") between instances of the aforementioned phases viewing, pausing and rating, comprising aspects such as taking on and off of the HMD, scene changes or between-video switches, and carrying out rating tasks within the HMD, i.e. with the scale shown on its screen, or on paper without wearing the HMD.
The simulator-sickness contributions of the different components are integrated to the overall simulator sickness score in model component "I" (cf. Fig. 1). To develop this rather complex model, we currently conduct research including a series of subjective tests to analyze each factor specifically and in terms of interactions. In this paper, as a first step, the evolution of simulator sickness over time is considered.

EXPERIMENTS
In this study, 360 • videos were presented to participants in full resolution. After each video, participants were asked to rate video quality (to align this research with related 360 • -video research) followed by a comfort question that reports their viewing comfort level. To enable short-term ratings in relation to simulator sickness, comfort ratings were used. As previously indicated e.g. by Singla et al. in [6], single-scale ratings can well reflect the overall effect of simulator sickness. The question was framed to promote positive outlook by asking for comfort instead of discomfort.
Eighteen different omnidirectional source videos with audio were used in the test 1 . The video sequences were downloaded from YouTube/Arte in 4K (3840×1920) resolution in highest available quality. The duration of each video sequence is 60 s to provide sufficient time for the subjects to explore the scene and possibly develop symptoms of simulator sickness. The videos were re-encoded using FFmpeg 2 with libx265 and two-pass (preset slow) encoding in two bit-rates (1 Mbit/s and 15 Mbit/s). These bit-rates were chosen to investigate the impact of bit-rates on comfort level. The source videos were selected to include some sequences expected to make the user uncomfortable, e.g. due to high amounts of motion. The 360 • videos were viewed on the HTC Vive Pro using the Whirligig player. The Field of View (FOV) was set to 360 and projection type to Barrel. The resolution of the HMD is 2880×1600 and its FOV is 110 • horizontally. The HMD was connected to a desktop PC with current CPU (Intel i7, 6700), NVIDIA GTX1080 graphics card and SSD. To record the viewing direction (yaw, pitch and roll) of the participants, an open-source framework 3 was used.
The Absolute Category Rating (ACR) (ITU-T Rec. P.910) test method was selected to evaluate the audiovisual quality. For comfort, a 5-point scale was used according to ITU-R BT.2021. 28 participants (14 female, 14 male) participated in the test. Visual acuity (20/25, Snellen) and color blindness (Ishihara) were tested. The test was preceded by a training session for test-subject familiarization. Each subject had to rate 36 Processed Video Sequences (PVSs, 18 sequences in 2 bitrates) over three test sessions. First, the participant had to watch the video for 60 s in an HMD and had 6 s each to provide the quality and comfort scores. Before and after each test session, participants filled the full Simulator Sickness Questionnaire (SSQ) [1].

RESULTS AND TEMPORAL EVOLUTION MODEL
Outlier detection to check the reliability of test participants was performed based on ITU-R Rec. BT.500-13, and no outliers were found. The analysis of quality scores is out of scope for this paper and will be addressed in follow-up research. The participants filled in the comfort question after every PVS. A scale inversion was carried out by subtracting the comfort scores from the value 6. The resulting current discomfort score (as indicator of simulator sickness) is assumed to primarily depend on the specific video sequence s k and time index t i of viewing it. To address temporal evolution in this paper, the discomfort scores for sequences with the same time index t i were averaged, and are shown in Fig. 2. Results for the three sessions are depicted in Fig. 2 (a)-(c). Fig. 2 (d) shows the results for the entire test. Here, the red lines indicate the point in time where the subjects were taking a break between two test sessions.
We used simple linear regression with regard to viewing time t i for our first model of discomfort scores. Here, it is assumed that discomfort increases over time and is gradually reduced during breaks between sessions. Linear regression hence was applied separately on each test session, see plots (a)-(c). To evaluate the performance of the proposed model, the Pearson Linear Correlation Coefficient (PLCC) between subjective scores and SiSiMo estimates was calculated. The resulting PLCC values for sessions 1, 2 and 3 are 0.68, 0.40 and 0.84, respectively. The overall (green) and per-session 2 https://ffmpeg.org/ 3 https://github.com/acmmmsys/2018-AVTrack360 regression lines (blue) in (d) indicate a saw-tooth like pattern, with increasing discomfort per session as well as a slight increase across sessions, in spite of the breaks. An analysis of the relation between discomfort and the source video sequences revealed the known link between content and simulator sickness also discussed in [2,3,5]. More details on this will be presented in a follow-up paper. To find out the impact of video bitrate on discomfort, a repeated-measures ANOVA was carried out on the individual ratings, indicating a statistically significant contribution (p < 0.01).

CONCLUSION
In this paper, we propose a conceptual Simulator Sickness Model SiSiMo able to predict sickness over longer viewing sessions. First results show that temporal evolution of discomfort, as a simplified measure of simulator sickness, can be estimated using linear regression over viewing-time index for 360 • videos. The performance of our model is promising, however with moderate prediction accuracy for some of the sessions (1 and 3). Here, additional consideration of the actual video viewed and the individual proneness to simulator sickness will be included for future improvements. In general, SiSiMo could alleviate the concerns of consumers about sickness and could be used to continuously analyze the comfort level of users while watching videos in an HMD. In our ongoing research, we started to consider physiological measurements like Electrodermal Activity (EDA) as complementary and more continuous information source about simulator sickness. Moreover, beyond the role of individual videos dedicated research will address the decay of simulator sickness during breaks and contributions of adaptations between phases. Here, HMD-based 360 • video will serve as a controlled starting point for further research on modelling simulator sickness.