Simultaneous Emotional Stimuli Prolong the Timing of Vibrotactile Events

Temporal information plays a crucial role in human everyday life. Yet, perceived time is subject to distortions. Emotion, for instance, is a powerful time modulator in that emotional events are perceived longer than neutral events of the same length. However, it is unknown how exposure to emotional stimuli influences the time perception of a simultaneous neutral tactile event. To fill this gap, we tested the effect of emotional auditory sounds on the perception of neutral vibrotactile feedback. We used neutral and emotional (i.e., pleasant-high arousal, pleasant-low arousal, unpleasant-high arousal, and unpleasant-low arousal) auditory stimuli from the International Digitized Sound System (IADS). Tactile information was a vibrotactile stimulus at a fixed intensity and presented through a custom-made vibrotactile sleeve. Participants listened to auditory stimuli which were temporally coupled with vibrotactile stimulation for 2,3,4, or 5 s. Their task was to focus on the duration of vibrotactile information and reproduce elapsed time. We tested the effects of valence and arousal of auditory stimuli on the perceived duration of vibrotactile information. Simultaneously presented auditory stimuli, in general, lengthened the perceived duration of the neutral vibrotactile information compared to neutral auditory stimuli. We conclude that emotional events influence time perception of simultaneous neutral haptic events.


Simultaneous Emotional Stimuli Prolong the Timing of Vibrotactile Events
Müge Cavdan , Bora Celebi , and Knut Drewing Abstract-Temporal information plays a crucial role in human everyday life.Yet, perceived time is subject to distortions.Emotion, for instance, is a powerful time modulator in that emotional events are perceived longer than neutral events of the same length.However, it is unknown how exposure to emotional stimuli influences the time perception of a simultaneous neutral tactile event.
To fill this gap, we tested the effect of emotional auditory sounds on the perception of neutral vibrotactile feedback.We used neutral and emotional (i.e., pleasant-high arousal, pleasant-low arousal, unpleasant-high arousal, and unpleasant-low arousal) auditory stimuli from the International Digitized Sound System (IADS).Tactile information was a vibrotactile stimulus at a fixed intensity and presented through a custom-made vibrotactile sleeve.Participants listened to auditory stimuli which were temporally coupled with vibrotactile stimulation for 2,3,4, or 5 s.Their task was to focus on the duration of vibrotactile information and reproduce elapsed time.We tested the effects of valence and arousal of auditory stimuli on the perceived duration of vibrotactile information.Simultaneously presented auditory stimuli, in general, lengthened the perceived duration of the neutral vibrotactile information compared to neutral auditory stimuli.We conclude that emotional events influence time perception of simultaneous neutral haptic events.Index Terms-Time perception, haptics, arousal, valence, audition, vibrotactile feedback.

I. INTRODUCTION
T IME perception is of ubiquitous relevance for human be- havior.From a very basic temporal (motor) task such as crossing the street to rather complicated tasks like planning and scheduling the weekly activities, we rely on our sense of time.However, time perception is subject to distortions.Perhaps one of the most strongest and salient modulator of time is emotion.Time seems to change its pace whenever we experience an emotional stimulus.For instance, when we are stressed to meet a deadline, time starts to feel like flying by while it drags if you are waiting a loved one to arrive.This phenomenon is called emotional temporal distortion [1].
The last two decades witnessed an abundance of studies investigating the perception of the presentation time of different emotional stimuli (e.g., [2], [3], [4], [5]).For instance, Droit-Volet and her colleagues studied how time is perceived when presented with angry, happy, and sad facial expressions compared to neutral.They found that time feels stretched when emotional faces are presented.However, in our daily life we do not only directly interact with emotional stimuli, but we often also interact with neutral stimuli under different emotional influences.Also, we speculate that, the perception of a neutral event might be influenced by the current emotion of the perceiver.Here, we studied how concurrent emotional stimuli influence the time perception of a neutral event.
In the circumplex model of emotions, Russel describes emotion with two main dimensions: valence and arousal [6].Valence refers to the degree of the pleasantness ranging from pleasant to unpleasant in a continuum [6].Arousal is the level of excitement, the strength of an emotion with two opposite poles very calm and sleepiness (low arousal) to excitement (high arousal) [6].Research targeting the time perception of emotional stimuli mostly focused on valence or arousal.With regard to valence, both positive and negative stimuli are perceived to last longer than neutral stimuli-often with a larger time-stretching effect for negative compared to positive stimuli [2], [7], [8], [9] exceptions from this pattern in [10], [11].
Also arousal plays a key role in emotional temporal distortions.These have been mainly demonstrated by focusing on physiological arousal in both humans and non-human animals.In pharmacological studies, non-human animals were administered psychoactive substances such as cocaine, methamphetamine, or nicotine, which modulate the level of dopamine in the brain, and thus physiological arousal; for animals general effects of arousal on timing were observed [12], [13], [14].In humans, arousal was measured by self-reports and physiological changes (e.g., heart rate) in the context of time perception.Temporal overestimations were observed with an increase in perceived or physiologically measured bodily arousal [5], [9], [15], [16].However, similar to valence, most studies with humans measured the temporal judgements on arousing emotional events (e.g., [5], [16], [9], [17]), whereas the role of arousal for neutral events has mostly been neglected in human studies (see [15] for one exception).
Finally, a few studies examined both valence and arousal dimensions and tested how they interact in terms of time perception of visual and auditory stimuli [2], [9].Angrilli et al. [16] selected standardized photographs from the International Affective Picture System (IAPS; [18]) in five categories: neutral, pleasant-high arousing, pleasant-low arousing, unpleasant-high arousing, and unpleasant-low arousing.They tested time perception using a reproduction and an analogue scale estimation in vision: The duration of negative images was judged shorter than that of positive images for low arousal stimuli while the duration of negative images was judged longer than that of positive images for high arousal stimuli.Using a similar design, Noulhaine et al., [2] conducted a study with emotional auditory stimuli [9] from the International Affective Digitized Sound System (IADS [19]).The results showed significant effects of valence and arousal.Specifically, negative sounds were judged to last longer than positive sounds and high-arousing sounds were perceived to last shorter than low-arousing sounds.At first glance, high and low arousing emotional stimuli might be expected to lengthen perceived durations compared to the neutral stimuli.However, the effects of arousal and valence on the timing literature provide conflicting evidence.Namely, the interaction between arousal and valence influenced time perception in vision [9] while the main effects of arousal and valence influenced the time perception in audition [2].The differences might origin from differences in processing across modalities [20], [21], [22] or the type of stimuli that were used.
Many authors discuss their results on emotional temporal distortion with respect to the scalar timing theory [23], [24].This theory suggest an internal clock that consists of three components: a pacemaker, a switch, and an accumulator.The pacemaker frequently emits pulses and sends them to the accumulator through the switch.The switch signals the start and end time of events.The accumulator counts the pulses between the start and stop of the switch.It had been suggested that both arousal and attention influence the internal clock [25].Specifically, the pacemaker speeds up with higher arousal and more pulses are emitted which results in time expansion [17], [26], [27].Further, others suggest that functioning of the switch requires attention -when attention is distracted from time perception, the switch works not properly [31] resulting in lost pulses and shorter time perception.
Thus far, studies mainly focused on the time perception of emotional events themselves.However, in daily life, the source of an emotional event and a target of time estimation do not necessarily be the same.For instance, in a soccer match, the source of emotion can be a shouting hooligan while a goal keeper tries to simultaneously estimate how long the ball took to travel a certain distance.When the events (i.e., cues) are causally linked together as in the example, they are more likely to be integrated whereas, in the absence of a causal link they could be segregated.Nevertheless, the effect of emotion would be reflected in the duration judgements.Despite of the daily relevance, the influences of simultaneous emotional stimuli on neutral events has rather been neglected in the literature.Here, using the tactile sense, we investigated whether auditory emotional stimuli modulate the time perception of simultaneous neutral tactile events.We selected neutral and emotional auditory stimuli from the International Affective Digitized Sound system [19] and coupled them with vibrotactile feedback which was presented through a custom-made haptic sleeve.Affective sounds were selected to manipulate both arousal and valence dimensions.We expected vibrotactile stimuli accompanied by emotional sounds are reproduced by longer vibrotactile stimuli compared to those accompanied by neutral sounds.We further speculated that the valence (pleasant vs. unpleasant) and the extent of arousal (high vs. low) of the stimuli would modify the amount of lengthening.

A. Participants
Power analysis was used to determine the sample size.The required sample size was calculated based on a medium effect (f : 0.253), 80% power, and alpha 0.05.The projected sample size was 27 for a repeated-measures ANOVA [28].Accordingly, 27 participants (16 females, age range: 21-42, mean age: 26 years 3 months) recruited and were compensated 8 €/hour for their time.The study was ethically approved by LEK FB 06 in accordance with the declaration of Helsinki [29] excluding the preregistration.Participants gave written informed consent prior to the experiment.

B. Apparatus and Setup
A custom-made vibrotactile sleeve is used to deliver vibrotactile feedback.The sleeve (Fig. 1) is developed to transmit multimodal haptic feedback on the forearm.It consists of vibrotactile and tapping actuators.In the current experiment, only one vibrotactile actuator was used to provide information (see Fig. 1 red square).A cylindrical eccentric rotating mass (ERM) actuator (VZ7AL2B1690002; Vybronics Ltd.) was placed in a custom-made encapsulation.Specifications of the vibrotactile actuator was as follows: 7 mm diameter, 16.5 mm body, 2.2 ∼ 3.6 operating voltage, 250 mA maximum current, 7.3 gravitational (G) maximum force.A DRV2605L motor driver (Texas Instruments) drives the vibrotactile actuator.The driver was controlled using an Arduino Uno Revision 3 with a custom-made script.Control commands were transferred from the main experiment script using a serial port communication.
Active noise cancelling headphones (Sennheiser HD 4.5 BTNC) were used to eliminate the sounds caused by the actuators and to present the sound stimuli.Using Psychtoolbox routines [30] the experiment was programmed in MATLAB 2021b.A standard keyboard was used for the reproduction task.

C. Stimuli, Design, and Procedure
The emotional auditory stimuli were selected based on [2] including the stimuli durations.Thirty-six IADS sounds were selected from five categories: pleasant-high arousal (e.g., rock and roll music), pleasant-low arousal (e.g., giggling), unpleasanthigh arousal (e.g., sirens), and unpleasant-low arousal (e.g., bees), and neutral sounds (e.g., street sounds).Each emotional sound category consisted of 6 sounds while the neutral category consisted of 12 sounds.Although the original length of each sound is 6 s, we only presented them for 2,3,4 or 5 sec in the current experiment.The sounds were played from beginning and cut from the end (e.g., from beginning till 2 s).Note that the shorter durations (i.e., 2 s, 3s, 4s, and 5 s) found to be associated with similar valences and arousals as the original IADS sound duration (i.e., 6 s) [2].
Vibrotactile stimuli were defined by vibration amplitude (acceleration) 2.52 m/s 2 and 85 Hz frequency which was presented to the lower back of the right hand (i.e., in between carpal bones and metacarpal bones, cf.Fig. 1).The intensity and location of the vibration was kept constant throughout the experiment.
A 4 (4 durations) × 5 (emotional content: pleasant-high arousal, unpleasant-high arousal, pleasant-low, unpleasant-low, and neutral) within-subjects design was used in a time reproduction experiment.The participants' task was to reproduce the duration of a vibrotactile stimulus that was accompanied by an emotional sound while they receive vibration to their right hand.We used combined stimuli of four durations: 2, 3, 4, and 5 s.The experiment consisted of 3 blocks and in each block 48 vibrotactile-sound pairs were presented (144 trials overall: 3 repetitions to keep the experiment short enough to avoid possible habituation and lengthening bias [31]).Each block consisted of two sounds from each, pleasant-high, unpleasant-high, pleasant-low, and unpleasant-low arousal categories and four of the neutral sounds presented in each duration.The order of the trials were randomized across the whole experiment.
In each single trial, a vibrotactile stimulus and an emotional sound stimulus were simultaneously presented (Fig. 2).Participants were instructed to focus on the length of the vibrotactile stimulus.After the stimulus presentation had ended, there was a delay of 1 s duration followed by the reproduction task.Next, a vibrotactile stimulus (without a sound) was presented on the same location.The participants were asked to press the space button with their left hand when they think that the elapsed time of the vibrotactile stimulus was equal to the duration of the first one.This was followed by an interval of random duration (2-3 sec).Then, the next trial started.
Before the experiment, participants were instructed about the task and performed five practice trials with neutral sound stimuli to familiarize with the experimental procedures.There were breaks of two minutes between experimental blocks.In total, the experiment took approximately 45 minutes.

III. RESULTS
Time reproduction performance was calculated per trial with a widely used T-corrected score [2], [9], [10], [26]: While T-estimated is the reproduction duration of the participant (i.e., perceived duration), T-standard is the physical vibrotactile feedback duration.The calculated values provide information on the relative extent and the direction of the temporal reproduction error.Negative values imply that the reproduced duration was shorter than the standard duration while positive values imply that the reproduced duration was longer than the standard duration.All analyses were conducted in SPSS 25 software (SPSS inc.) and figures were plotted in Matlab R2021b (MathWorks inc.).Greenhouse-Geisser correction was applied whenever the sphericity assumption is violated.
Post-hoc paired comparisons between the levels of valence revealed significantly shorter reproduction times for neutral (M = -.10,SE = .04)and pleasant (M = -.08,SE = .04)as compared to unpleasant contents (M = -.04,SE = .04;Bonferroni corrected p < .05).Post-hoc t-tests between durations Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
Further post-hoc t-tests, clarifying the interaction, revealed that the reproduction of vibrotactile feedback was longer when coupled with unpleasant compared to pleasant and neutral sounds for a 2 s duration (p = .048,p = .003respectively: Bonferroni-corrected for all comparisons).For 4 s and 5 s durations, reproduction was longer for the unpleasant compared to neutral auditory coupling only (p = .048,p = .002sequentially).Finally, for the 3 s duration no significant differences between auditory stimuli was observed (all p >.05).

B. Arousal and Duration Effects on Vibrotactile Time Reproduction
In order to test the effects of arousal and duration on vibrotactile time reproduction, we averaged the individual T-corrected values across valence separately for each level of arousal and duration (Fig. 3(b)).The 3 (arousal: high, low, and neutral) × 4 (duration: 2, 3, 4, and 5 s) ANOVA on T-corrected vibrotactile time reproduction revealed significant main effects of arousal and duration: F(1.52, 39.64) = 5.53, ηp 2 = .18,p = .01,F(1.31, 33.94) = 49.05,ηp 2 = .64,p < .001,respectively.The interaction between arousal and duration was not statistically significant, F(3.57, 92.81) = 1.88, ηp 2 = .07,p = .13.The main effect of duration only repeats what we already analyzed in Section III-A.However, pair-wise comparisons between the levels of arousal showed that time was reproduced significantly longer in the low arousal (M = -.06,SE = .04)as compared to the neutral condition (M = -.10,SE = .04).The high arousal condition did not significantly differ from neither of the other two conditions (overall a-level = 5%).

C. The Interaction Between Valence, Arousal, and Duration in Vibrotactile Time Reproduction
For the completeness, we performed a 4 (duration : 2, 3, 4, and 5 s) × 2 (valence: unpleasant and pleasant) × 2 (arousal: high and low) factorial ANOVA to test the interaction between arousal and valence as well as the interaction between duration, arousal, and valence (Fig. 4).As the main effects would be redundant with A and B, we only report the interactions.Neither the interaction between valence and arousal, F(1, 26) = .85,ηp 2 = .03nor the interaction between arousal, valence, and duration, F(2.32, 60.12) = .44,ηp 2 = .02,were statistically significant.

D. The Relationship Between Physical Sound Features and Reproduction
Not only the semantic properties of the sounds, but also low-level features such as pitch or frequency could influence time perception.To test for possible effects, we calculated correlations between sound features (peak amplitude, average RMS [root mean square] dB, dominant frequency, and dominant pitch) and the averaged T-corrected values across durations for each sound.We determined the dominant frequency and pitch frequency, which were missing in the IADS database, using fast Fourier transform (FFT) with Phyton scipy.fftfunction while we used peak amplitude and average RMS dB measures from the IADS database.As dominant frequency, we determined the frequency with the highest amplitude and as the pitch frequency calculated the maximum amplitude for different windows of the signal (185 ms) and averaged across all.Spearman's rho results did not show significant relationships between t-corrected values and any of the physical measures (peak amplitude dB: r s = -.23,p = .18;average RMS dB: r s = .03,p = .86;dominant frequency: r s = -.06,p = .71;and dominant pitch: r s = -.08,p = .63).

IV. DISCUSSION
We investigated how an emotional auditory stimulus which is accompanied by a neutral vibrotactile information influences the reproduction of that neutral event.We found that both arousal and valence of the auditory stimuli modulated the perceived duration of the neutral tactile information: Unpleasant sounds significantly lengthened the reproduction compared to neutral and pleasant sounds.Further, Low arousal sounds resulted in significantly longer reproduction of the neutral events.Note also that an analysis of the interaction of valence and arousal did not yield significant effects.Overall, we conclude that emotional stimuli, and in particular unpleasant and low arousing ones lengthen the perceived time of simultaneous neutral events.
Noulhiane et al. [2] showed that both valence and arousal of the auditory stimuli influence temporal judgements of the auditory stimuli themselves.We extended their findings by showing that not only the timing of the emotional stimuli in question but also the timing of a simultaneously presented neutral stimulus in another modality is influenced by the emotional stimulus.Similar to [2], simultaneously presented unpleasant sounds resulted in longer vibrotactile reproduction compared to pleasant sounds.Unlike [2] high-arousing sounds did not cause significantly shorter reproduction compared to low-arousal sounds.
However, low-arousing sounds lengthened the vibrotactile reproduction compared to neutral sounds.Taken together, results from the current study show that valence and arousal of a simultaneously presented emotional stimulus to the auditory modality can change the time perception of a concurrent haptic event.
Unpleasant sounds lengthened perceived time compared to neutral and pleasant sounds.This is in line with previous research on perceived durations of emotional faces [32], [33], sounds [2], [5], and photographs [9].From an evolutionary point of view, it is crucial to escape from a danger for survival, whereas engaging in a pleasant activity is not as important for existence.It seems plausible, that unpleasant sounds rather link to the need of avoidance behavior, whereas pleasant sounds are associated more with appetitive behavior.Thus, unpleasant sounds could prepare avoidance behavior, which might benefit from subjective time stretch.In contrast, pleasant sounds might prepare appetitive behavior and time stretch is less relevant.In line with this assumption previous studies have shown that, compared to positive and neutral valence, negative valence yields stronger cognitive, social, and physiological responses which is called as negativity bias [34], [35].Also in line, during near-death experiences (actual fear of death) such as car accidents (frightening events), people experience time slowing [36].Perhaps, the stretched time perception at the moment of vital danger helps us to take an appropriate action.
The reproduction performance followed Vierordt's law [37] (see [38] for a review).Namely, shorter durations were overestimated (positive T-corrected values) whereas longer durations were underestimated (negative T-corrected values).The durations here were rather longer, hence underestimated.In the detailed analysis the difference between unpleasant and neutral conditions reached significance for 3 out of 4 durations, suggesting a quite general effect of unpleasant sounds.However, time was significantly lengthened by unpleasant sounds relative to pleasant ones, only for short sound duration of 2 seconds, but not for the longer ones.This lengthening might suggest that emotional reactions to negative events are faster or based on less sensory input -since they are relevant for survival -as compared to that to positive events.Also arousal of the auditory stimuli lengthened the perceived duration of accompanying tactile events.In particular, low arousing sounds yielded significantly longer reproductions than neutral sounds, whereas the effect did not reach significance for high-arousing sounds.Arousal has been suggested to speed up the pacemaker in the internal clock, and thus to lengthen perceived time [17], [26], [27].This assumption well explains our effect of low arousal.However, it predicts even stronger effects of high arousal, which we did not find.Maybe, high-arousal sounds also influenced attentional processes in the internal clock that counteracted effects of pacemaker acceleration: Attentional models of time perception propose that the more attention is allocated to time judgments, the longer the subjective time.Specifically, the pacemaker switch (opens the gate to the accumulator) will close or open later and more frequently which would result in loss of pulses [39], [40].High-arousing emotional events might particularly distract from judging time, because attention would be directed to the emotional events.Hence, we might not see a strong overall effect from high-arousing sounds.
Here, we selected the auditory emotional stimuli from a database, and the stimuli have been standardized there with respect to valence and arousal.These stimuli have been proven to be effective not only in the standardization study [19] but also in a timing study [2].Therefore, we did not collect either physiological or self-report measures to test the individuals actual emotional reaction towards the auditory stimuli.In future, we plan to extend our studies by including both physiological (i.e., skin conductance and facial electromyography) and selfreport measures of arousal and valence in the context of time perception.One could argue that the vibration stimulus itself could induce different emotions (see [40], [41]).However, we kept the vibration stimuli constant across the whole experiment.Therefore, the differences we found can be directly related to the emotional change in auditory stimulus (i.e., the manipulation).
Similarly, low-level features of the auditory stimuli (e.g., amplitude or frequency) could influence the time perception of concurrent vibrotactile events [42].We did, however, not find evidence for this assumption when peak amplitude, average RMS dB, dominant frequency, and dominant pitch were correlated with the T-corrected reproduction results.It is important to note that, adaptation to a high frequency vibration is found to compress the perceived duration of a low frequency vibration [42].This should, however, not have affected the present results, because the vibration frequency was kept constant (85 Hz throughout) and the presentation order was fully randomized.Future studies can look at whether different frequencies influence the perception of emotional events.
Due to the nature of emotional stimuli, we only investigated the effect of emotion on neutral stimuli in time intervals in the second range (2-5 s).This was because presentation of the auditory stimuli under 2 s would reduce the understanding of the contextual emotion effects.Different mechanisms are involved in milliseconds to seconds range.Future studies could investigate the emotional influences on neutral haptic time perception in milliseconds range.

V. CONCLUSION
For the seconds range, we showed that the time perception of neutral haptic stimuli is influenced by simultaneously presented emotional stimuli in the auditory sense.These results provide important insights which can be used in several applications.When we try to meet a deadline or when our workload increases, time feels compressed which then leads to stress.In these stressful scenarios, lengthening the perceived time perception of an individual would not only reduce the stress but also increase the individual's well-being.For instance, using emotional stimuli in one modality could be used to lengthen time of a neutral event in another modality, especially in these stressful situations where the time dilation is needed [43].

Fig. 1 .
Fig. 1.The custom-made vibrotactile sleeve.Red square marks the actuator that is used to give vibrotactile feedback during the experiment.

Fig. 3 .
Fig. 3. Average T-corrected values per standard duration (2, 3, 4, and 5 sec): (a) As a function of valence (neutral, pleasant, and unpleasant) and (b) as a function of arousal (neutral, high, and low).Error bars represent standard of the mean.

Fig. 4 .
Fig. 4. T-correct values (M ± SD) per standard duration (2, 3, 4, and 5 s) as a function of emotional content (neutral, positive high, positive low, negative high, negative low).Error bars represent standard of the mean.