CoConUT: context collection for non-stationary user testing

CoConUT is an Android app for collecting the mobile context as well as the frequency of interactions during mobile field studies (for example usability studies) using sensor data on the test device. For evaluation purposes the recorded user trial sessions can be visually explored. This facilitates an assessment of the user's attention patterns and enables the detection of limited cognitive resources caused by distracting contextual factors. The app was tested in a preliminary study for technical feasibility and is planned to be extended in the near future.


Introduction
In the field of Mobile HCI, there is still no consensus whether to test mobile products in a lab environment or in the field [5,6,8]. In lab studies distractions and interruptions may be eliminated due to controllable surroundings. User tests in laboratory settings, however, lack the ecological validity of a test situation in the field.
This issue is of special importance if we are interested in gaining insight into context, which is crucial for studies about usability or user experience during mobile usage, since the influence of contextual data on Mobile Human-Computer-Interaction is huge [7,1]. For instance, on a street the user's attention towards the test device can be limited due to varying contexts demanding mental resources, as users have to also look out for their surroundings, especially during potentially harming situations like transfer (crossing a street, entering public transportation, etc.). This aspect may even become critical since paying attention to the wrong task can lead to physical harm. Especially in research about mobile notifications the consideration of mobile context is helpful to understand the user's main focus, as well as potential sources of interruptions.
In [9], Oulasvirta et al. have shown that main tasks (i.e. test tasks) and context-related tasks (e.g. paying attention to the surroundings) compete for the user's cognitive resources and can result in resource depletion regarding the main task. Due to this lack of focus the participants' behavior during the study can remain unclear during evaluation. Monitoring the user's focus and measuring certain context dimensions allow for a better insight into the user's attention patterns during the course of the study.
In order to get a better understanding of humancomputer interaction during field studies we have developed the CoConUT app for mobile field studies which will be presented as following: First, a very brief overview over related applications is given.
Subsequently, concept, design and implementation of the CoConUT app are being described. Furthermore, a preliminary evaluation and first insights into the data are depicted. We conclude with a summary and some directions for future work.

Related Work
The field of context aware systems is vast and growing continuously [3]. However, context aware apps for supporting mobile fields studies are rare. The application MyExperience e.g. collects objective and subjective in situ data like objective device usage through passive logging, user context through sensor reading and subjective user experience through feedback (see figure 1). It was implemented for Windows Mobile 2005 [2]. ContextPhone, a prototyping platform implemented for Symbian OS and Nokia Series 60 Smartphone platform, enables developers to access context as a resource, incorporate existing applications and offer fast interaction to their users as well as unobtrusiveness, among other possibilities (see figure  2) [10]. While apps for collecting contextual data of surroundings exist, most of them are outdated. However, since modern mobile devices support an increasing range of sensing possibilities, up-to-dateness and extensibility prove to be crucial for context collecting tools. For instance, Google's new app Science Journal 1 provides up-to-date sensing and recording capabilities, but is not open-source and thus nonextensible.
Concerning context modeling, Jumisko-Pyykkö et al. presented a descriptive model of context of use for 1 https://play.google.com/store/apps/details?id=com.google.andr oid.apps.forscience.whistlepunk ContextPhone [10] mobile HCI (the CoU-HMCI model) in which they summarized five components, their subcomponents and descriptive properties [4]. As a result from an extensive literature review they identified five main components (physical context, task context, social context, technical & information context and temporal context) as well as properties of mobile context (level of magnitude, dynamism, pattern and typical combinations). The design of CoConUT will closely follow this model, as described in the next section.

The CoConUT App for Mobile Field Studies
We have designed and developed the CoConUT app for mobile field studies to assess contextual dimensions and interaction data during field studies.
The application was built in Android Studio 1.4 with a minimal SDK 17 (Android 4.2) and Java Version 1.8.0 25, after a requirement-driven prototyping process.

Modeling Context
In our work we closely follow the aforementioned CoU-HMCI model of context of use for mobile HCI. In order to gain insights into all five main context components, we have identified a set of sensors and data which can be drawn upon. The particular sensors and how they map to the context dimensions of the CoU-HCMI model are described as follows: • Physical context: Encompasses the user's location, gradient and altitude, physical objects, orientation, weather and lighting conditions. CoConUT tracks movement and location by GPS. Brightness is measured by using the light sensor, level of noise by recording the amplitude with the microphone.
Weather is not assessed by CoConUT since it is accessible through external sources.
• Temporal context: Contains factors of past and future situations, from time of day to the week, month or season and time available. Time in CoConUT is measured by timestamps.
• Social context: Refers to other people present, their characteristics, their apparent roles, and interpersonal interaction. Since exact sensorial measurement of people nearby is technically challenging, the number of Bluetooth devices in the direct surroundings serves as an approximation of the social context in CoConUT. Other approaches like using the built-in camera to monitor persons nearby were omitted due to feasibility.
• Task context: This dimension refers to benefits (e.g. resources available), constraints (e.g. time pressure), multitasking and possible interruptions that are task-related. Those facts are already available to the operator or can be assessed through a post-study questionnaire.
• Technical context: Encompasses devices, available infrastructure, facts and system assumptions, sensors and network services. This information is covered by the test setup and thus known to the operator.
A detailed overview about sensorial data collected by CoConUT, externally available data and how they map to the context dimensions is also given in figure 3.

Measurement of interaction is realized by tracking the frequency of touch interactions on the touch screen.
Other means of interaction, like camera or voice input, were omitted due to low occurrence probability and  high CPU load. Also "blind" interactions (like e.g. pressing buttons or acceleration) were not being considered since those forms of interaction are lacking the user's focus on the smartphone.

Data Visualization
The operator can record single field study sessions during the trials and afterwards visualize them (see figure 4, 5 and 6). The sensor data to be measured can be selected and adjusted in the preferences.
The visualization aims at providing a first insight into the gathered data from different angles, exploring patterns and aiding at reasoning about the course of the study. On the basis of the data the operator is able to formulate tailored questions about certain points of time or location in a post-study exploratory interview, if questions arise (e.g. why there was no interaction in certain locations, or why users didn't manage to keep their pace).
Hence, for exploration purposes, the app features the following visualization modes: • Normalized overview of all sensors: All sensor data are displayed in a normalized way on a single timeline. The y-axis has a scale from 0 to 1, where 0 represents the lowest measured value (or simply zero) and 1 the highest (see figure 4).
• Single sensor: Data values of a single sensor over time, normalized on the y-axis by the highest value (see figure 6).
• Ratio: A tendency between the normalized data values of two sensors (see figure 4).
• Map: Data values of a single sensor are displayed in a normalized way on a map. The color palette ranges from green over yellow to red, where red represents the highest value, green the lowest and yellow values in the mean (see figure 5).

Evaluation
We did two studies to test technical setup and general feasibility and to gather some first exploratory data. Each of the study sessions approximately lasted 20-30 minutes per participant.
In the first study, six students (3f/3m) between 20 and 24 years took part. The participants were being asked to walk a certain route in the city of Vienna while performing search operations in the web browser on an Android device. In the background the device gathered data using CoConUT. The walking route consisted of more or less crowded parts (shopping area without traffic or side streets with little traffic). The second study featured nine students (5f/4m) between 20 and 25 years. The walking route was located in the Viennese university district and featured regular streets with medium traffic, but had to be walked along twice: participants in the second study performed the same search operations in the browser but were also instructed to walk the route again while playing the game Angry Birds 2 . In both studies participants were explicitly instructed to pay attention while crossing streets to avoid critical situations.
Important preliminary feedback on the app was assessed during the studies and the general technical setup proved to be feasible. Due to low participant number the results might be limited though. Overall, the data collected by the app allows for first exploratory glimpses and enables the operator to get an overview of the course of the study. Both studies for example showed that attention on the device, measured by interaction frequency, varied enormously during regular walking speed. Users apparently did not have to slow down due to crowdedness or traffic while solving the given tasks. In the second study the game Angry Birds needed less, but more complex user input compared to the web search task.
Further exploration of these findings will be pursued in studies which we are currently planning.

Conclusion and Future Work
We presented the app CoConUT for data collection in mobile field trials, including data about mobile context and frequency of interaction. The resulting interaction patterns indicate the user's attention on the device during mobile field studies. The collected data can be visualized on the smartphone to enable visual exploration for each participant by the operator immediately after the study.
Future work will encompass various advances regarding recording, visualizing and analyzing mobile field study data with CoConUT: Currently we are working on a visualization dashboard for display on a regular PC to enhance explorability of the data sets by the operator and allow for detailed, visually aided analysis. The operator will have the possibility to cluster data over more than one participant and combine or compare the data sets.
Furthermore, we aim at further investigating mobile context by developing a complementary headpiece for assessing mobile surroundings. The participant's face and screen interaction will be recorded on video. Additional sensors will monitor the user's stress level, walking patterns and eye focus on the device.