Benchmarking data on worker reactions to triggering events
Authors/Creators
- 1. New York University
Contributors
Data collector:
Research group (4):
Supervisor (2):
- 1. New York University
Description
1. Real-world Benchmarking Data
The objective of this task was to determine if Virtual Reality-based captured behavioral data on responses to notifications are similar to what is expected in real-world settings. For this purpose, a real-world bench mark experiment was designed to capture participant response times to wearable watch alarms triggered upon simulated traffic near the mobile work zone on the experiment site in an urban setting. The proposed scope of data collection of the real-world study included the external environmental factors (e.g., site accessibility, weather). The key parameters of research are defined as reaction time to received alarms and the heart rate measures. Table 1 provides the list of parameters that were controlled and measured during the experiments.
| Variable name | Descriptions | |
| Key parameters captured | Reaction time | The time that one takes from getting the haptic or sound alarm from a wearable alarm device, herein referring to the apple watch, to the point when the participant gives a response by stopping the alarm by pressing on the screen of the smartwatch |
| Inter-beat interval (IBI, heart rate) | The time interval between individual beats of the heart; the data is measured by using E4 application provided by Empatica | |
| External factors tracked | Ambient noise | The level of ambient noise in the area is a factor potentially influencing participants’ reactions and is considered in the experiment design |
| Temperature | Daytime temperature recorded at each experiment | |
| Number of pedestrians on site | Number of participants counted during the time of the experiment to record on the varying factors in the external environment in real-world settings |
In the experiment, each participant was asked to participate in the experiment three times. In each trial, data was recorded separately for each alarm sent to smartwatch from the administrator at triggering events (precisely, every time the remote-controlled toy car reaches the line 30 ft apart from the designated work area). Each alarm signal at each trial was recorded for all 31 participants to the experiment. Timestamps are automatically recorded in server in the events recorded in the format of Table 2:
| Timestamp | From | Event | |
| 0 | 2022-12-08 13:37:53.101391 | VR | Received car approaching alert, mode=3, id=1000 |
| 1 | 2022-12-08 15:53:05.098288 | Watch | Start Simulation |
| 2 | 2022-12-08 15:53:07.437488 | VR | Received car approaching alert, mode=4, id=1004 |
| 3 | 2022-12-08 15:53:13.064067 | Watch | Stop Simulation |
| 4 | 2022-12-08 15:53:13.163635 | Watch | Stop Simulation |
| ... | |||
| 2417 | 2023-03-03 16:17:46.166644 | Watch | 1398 |
| 2418 | 2023-03-03 16:18:00.004425 | Watch | 1398 |
| 2419 | 2023-03-03 16:18.01.272071 | Watch | 1398 |
| 2420 | 2023-03-03 16:18:07.359187 | Watch | Stop Simulation |
| 2421 | 2023-03-03 16:18:07.388183 | Watch | Stop Simulation |
Some intervals used different timestamps as benchmarks to calibrate on the vehicle speed and user response time to the alarm signals, which include the following cases:
1) At the beginning of each trial, vehicle travels 70 ft from start point to the 30 ft apart point, when the first alarm is signaled; given this travel distance, the travel time of the first trip the toy vehicle makes is calculated by subtracting tn_alarm1_sent from tn_start.
2) Similarly, user response times to all alarms are recorded by subtracting the timestamps when the alarm is received by participant from when the alarm is sent from the server. (tn_alarmn_sent - tn_alarmn_received)
2. Supplementary Data
Ambient noise level data were collected using a noise meter, allowing to save noise level by seconds to multiple seconds (i.e., 5, 10, 30, 60 seconds). All noise data recorded were recorded in the interval of one second using the meter.
The collected data was processed to match the certain timestamps collected for user response time data collected in the experiment to allow comparisons and correlation analysis to be performed later on, which include the following: 1) worker response; 2) sending of alarm signals; 3) start and stop of experiments.
All data points were later modified using the rolling mean function of pandas python module to replace the missing data points by moving average method.