What cognitive and affective states should technology monitor to support learning?

This paper discusses self-efficacy, curiosity, and reflectivity as cognitive and affective states that are critical to learning but are overlooked in the context of affect-aware technology for learning. This discussion sits within the opportunities offered by the weDRAW project aiming at an embodied approach to the design of technology to support exploration and learning of mathematical concepts. We first review existing literature to clarify how the three states facilitate learning and how, if not supported, they may instead hinder learning. We then review the literature to understand how bodily expressions communicate these states and how technology could be used to monitor them. We conclude by presenting initial movement cues currently explored in the context of weDRAW.


INTRODUCTION
Cognitive and affective states play as important a role in learning and learning outcomes as they do in other aspects of life. It is thus necessary to take them into consideration when designing technology-supported or -mediated learning. Indeed, technology that is able to monitor such states would be better at providing tailored content, activities, and intervention to facilitate learning.
Although there have been several studies that have investigated automatic monitoring of cognitive and affective states in the context of learning (e.g. [ [8]), a gap still exists in understanding which of these states is of benefit to monitor. This paper is a first step in providing such understanding through discussion of three relevant states: selfefficacy, curiosity, and reflectivity. We focus on these states as they have not received much attention (in the affective computing community) compared to other learning states such as frustration, concentration, and boredom. For example, beyond studies such as [4][5] [6], there has been limited discussion on learning facilitation opportunities that could be created in monitoring self-efficacy in digital learning systems. Although [9] provide an elaborate discussion on curiosity, our discussion is different in that we focus on the benefit of tailoring learning support to this state in a digital learning environment. Unlike self-efficacy and curiosity, reflectivity has barely received any attention in learning-related computing studies. A wider range of states (including states that emerge in the context of social interactions in the classroom, e.g. embarrassment) will be considered in future work. We set our discussion of self-efficacy, curiosity, and reflectivity in the context of the weDRAW project [10] that takes a multisensory and multimodal approach to support learning. It focuses on learning of mathematical concepts in primary school children through bodily exploration of these concepts. The embodied approach taken is based on literature that show that cognition in the context of learning and learning itself is embodied [11][12] [13]. Findings show that one of the roles of the body in this context is to facilitate the analysis (i.e. exploring, processing, and explaining) of abstract concepts in a physical world [13].
Bodily gestures exhibited during learning can also be thought of as physical bodies (that become study materials for learning the enacted concepts) to abstract concepts [13]. Given the importance of bodily gestures and body movements, therefore, we further discuss its use as modality for automatically detecting these states in the setting of the weDRAW system and describe ongoing work.
The rest of the paper is divided into 3 main sections. In Section 2, we discuss the significance of self-efficacy, curiosity, and reflectivity, the utility of monitoring them in the use of digital learning systems, and the possibility of monitoring them based on bodily gesture and body movement cues. In Section 3, we briefly describe ongoing work in the area towards the development of the weDRAW system. A conclusion is given in Section 4.

ADDRESSING SELF-EFFICACY, CURIOSITY, AND REFLECTIVITY
In the context of learning, self-efficacy, curiosity, and reflectivity are epistemic states, i.e. related to cognitive appraisal or processing of information being learnt, different from achievement states (activity or outcome related), topic related states, and incidental states which emerge from experiences outside the classroom [14]. Thus, they are central to learning and it is critical to understand how they may be promoted, moderated, or supported in digital learning environment. In this section, we first discuss their influence on learning; we then discuss the opportunities that monitoring them opens up to facilitate learning. We conclude with a discussion of how such monitoring may be done based on bodily cues.

2.1
The Influence of Self-Efficacy, Curiosity, and Reflectivity on Learning The importance of self-efficacy in learning and learning activities is in its influence on the amount of effort and the level of persistence that a learner will put into the completion of the learning task in the face of barriers [15]. [16], for example, showed that children with higher level self-efficacy for arithmetic problem solving spent a significantly greater amount of time attempting the problem than those with lower levels. [17] further theorise that the effect of self-efficacy is not on behavioural engagement (effort and persistence) alone but that it may also have an effect on cognitive engagement (e.g. in reflection, the use of helpful strategies) and motivational engagement (e.g. in interest, enjoyment). Self-efficacy also has an influence on curiosity to learn as was found in a study by [18] with undergraduate students in a trivia task. The authors found that in cases where participants indicated not knowing the answer or that it was on the tip of their tongue, the reported level of confidence in the ability to identify the correct answer if given multiple options predicted the level of curiosity for the correct answer. Curiosity itself is an important state in learning as it directly affects information seeking behaviour [19] [20]. For example, [18] found higher levels of curiosity about the correct trivia answers to be significantly positively correlated with efforts made to retrieve these answers. There have also been findings suggesting that curiosity further facilitates acquisition of knowledge. A study by [20] showed that regions of the brain linked to memory and learning were activated in those who reported higher levels of curiosity about the correct answers in a trivia task. In a follow-up study where participants were shown the correct trivia answers and were afterward invited to re-take the quiz after a fortnight, participants who had reported higher levels of curiosity (and had memory regions of the brain activated) were found to show significantly better recall of the correct answers.
The cognitive strategy used in problem solving, reflectivity versus impulsivity, also affects learning outcome. In contrast to impulsivity, reflectivity enables learning and problem solving as it stimulates focus of attention, more analytical cognition, and use of helpful problem-solving strategies [21]. Indeed, in a study by [22], children who were impulsive performed significantly worse in arithmetic problem-solving tasks than their reflective peers. Impulsivity may be beneficial for tasks that require holistic processing [21] as was found in [23] where impulsive children were significantly more time efficient (and not inferior in accuracy) in a global matching task than the reflective.

Opportunities in Monitoring Self-Efficacy, Curiosity, and Reflectivity in Learning
It has been shown in the previous section that self-efficacy is important in learning because it influences engagement constructs which in turn influence performance outcome of learning [16][17] [18]. Findings in [16] suggest that self-efficacybased intervention may indeed be a means of promoting engagement and so improving learning outcomes. In their study with arithmetic tasks, children were assigned to two intervention conditions. In one condition, they were taught by an adult who worked through example problems before they solved any problems themselves. In the second condition, the children were given the same lesson in written form to study on their own. Children in the former condition reported significantly higher increase in self-efficacy levels from before the lesson than those in the latter condition. This finding suggests that targeted interventions can promote self-efficacy. A further finding was that the increase in self-efficacy levels was significantly related to increase in persistence and also performance on the problems. This points to opportunities to facilitate learning through addressing self-efficacy. In fact, [17] conclude that because selfefficacy is easier to directly promote than behavioural, cognitive, and motivational engagement variables, it may be one of the most convenient means to promote learning outcomes. The effect of addressing self-efficacy is cumulative as improvement in performance further enhances self-efficacy. However, there may be times when it is necessary to moderate self-efficacy levels rather than promote it. This is because a pupil with wrong estimations of high level of self-efficacy for a concept that they have not previously encountered or fully understood may not pay as much attention as they should in a lesson on that concept [17]. There may also be intervention avenues for curiosity. For example, curiosity level is expected to increase with increase in knowledge. [19] theorize that the positive relationship between curiosity and knowledge is because when knowledge level is low, attention is focused on the known rather than the unknown, whereas increases in knowledge can switch the attention focus What Cognitive and Affective States Should Technology … ISIAA'17, November 13, 2017, Glasgow, UK and so evoke curiosity. However, self-efficacy moderates this relationship between curiosity and knowledge. When knowledge self-efficacy is high, curiosity may not increase with further increase in knowledge even if there still remains a knowledge gap [19]. Findings in [18] suggest that self-efficacy may even play a mediation role in low levels of knowledge with higher levels of self-efficacy significantly leading to higher curiosity levels. This role of self-efficacy on state curiosity in this knowledge strata was found to be as strong as the role of trait curiosity [18]. [20] found that the relationship between curiosity and self-efficacy may actually be an inverted u-shape suggesting that low curiosity levels may be due to either very low or very high self-efficacy. This points to further need to understand the self-efficacy of a learner in order to be able to provide appropriate intervention to address their curiosity levels. Beyond increasing knowledge to bring attention to knowledge gap and so lay the foundation for curiosity, and further promoting confidence in the ability to close the knowledge gap to evoke curiosity, manipulation of the importance of new knowledge, saliency of the knowledge gap, and surprise may additionally be used to enhance curiosity [9].
Although reflectivity/impulsivity may be largely stable, it may still be of benefit to address it given its significance to learning and problem solving. Rather than attempting to promote increase in reflectivity, the intervention needed here may be coaching during problem solving to support and train pupils with a more impulsive cognitive style. It is possible that the same form of coaching for reflective children may be found patronising making the learning experience less challenging and enjoyable for them. It is, thus, important to tailor the intervention to the level of reflectivity of each child where possible. In [22], significant effect of reflectivity-based intervention on arithmetic task performance was found for impulsive children. The intervention was a problem-solving strategy training where the children were taught to reflect on and solve the problems in three ways: considering the numbers as tokens, then as pictures or sketches, and then as symbolic representation [22]. They were further taught to re-read the problem and highlight the actual task required in the problem statement [22]. Another strategy they were taught was dealing with large number problems by first attempting them using smaller numbers [22]. There was no significant effect of this intervention for reflective children. This group may possibly benefit from intervention that facilitates learning and problem solving in holistic tasks which they find more challenging than analytical problems [21] although they can perform well in them [23]. Self-efficacy level could be used to determine if such intervention is required.

The Body as a Modality of Learning Self-Efficacy, Curiosity, and Reflectivity
In the previous section, we highlighted opportunities for selfefficacy, curiosity, and reflectivity to be addressed in a learning environment so as to promote learning. The consequent need then is to understand how these states may be monitored in such environment. In this section, we briefly review how this has been previously explored and then discuss the possibilities of using bodily gesture and body movement cues for automatic detection.
Previous studies provide evidence of the feasibility of automatic detection of levels of self-efficacy and curiosity. For example, [4] used physiological signals and interaction data (such as time spent and progression towards the goal) for automatic detection of levels of self-efficacy. They obtained accuracy of 0.87, 0.83, 0.79, and 0.75 for detection of 2, 3, 4, and 5 levels respectively. Similarly, in [5], facial cues were used to automatically detect levels of self-efficacy with R 2 of 0.67 and 0.43 for middle school and college students respectively. In [6], a combination of facial cues, skin conductance, computer mouse handling, sitting posture cues, and interaction data (e.g. progression towards goal) were used to automatically classify 5 levels of self-efficacy with performance of 0.82 R 2 . In [8], facial muscle activity, skin conductance, and electrocardiography signals were used to automatically differentiate curiosity from engagement, confusion, frustration, delight, boredom, and neutral with F1 score of 0.36.
While these studies provide evidence of the possibility of detecting these states, we argue that bodily gestures and body movement are relevant and perhaps even more informative than physiological signals and facial expressions in this context. As discussed in the introductory section, this modality is a primary component of traditional learning environments and may be a window into the mind of a learner [12]. Further, unlike, the traditional affective computing modalities (face and voice), bodily gesture and body movement encapsulate information about the action tendency of the learner towards coping with or addressing the experienced state and so offer unique insight into subjective experiences [24]. The role of bodily gestures and movement cues may be enhanced in the weDRAW system where learning activities will be designed to involve this channel unlike the sedentary scenarios considered in previous studies [4][5][6] [8].
Relationships have indeed been found between bodily gesture and body movement behaviour and self-efficacy, curiosity, and reflectivity-impulsivity. For example, in [25], it was found that movement performance cues enabled observer assessment and automatic detection of levels of movement-related self-efficacy. Speed of movement, range of motion, muscle tension, dissymmetry in movement, and movement fluidity were particularly found to be useful cues. Although their work largely focused on a clinical population, some of these cues are similar to those found in [26] with child athletes performing gymnastic routines. [18] show that bodily behaviour may also be a useful cue for assessing curiosity. In their study, they found that those who reported higher level of curiosity about the correct answers in a trivia task were more likely to explore the answer packs given to them. Exploratory behaviour may, thus, be a possibly cue of this state. Movement performance has also been found to be related to reflectivity in [27] where reflective children were found to be better at motor tasks (using a racquet to hit a ball towards a target) than impulsive children.
In this section, we briefly present ongoing work towards the investigation of automatic detection of relevant learning states (such as self-efficacy, curiosity, and reflectivity) from bodily cues.
There are two main learning/problem-solving scenarios that are currently being focused on in the weDRAW project to investigate how these states may be expressed bodily. One scenario is Kinect-based arithmetic problem games where children interact with a digital world through gestures or movement. In the second scenario, children solve arithmetic problems in the natural world using the same modality. The Kinect is also used in this scenario but only as a (body movement) sensor; wearable inertia sensors are additionally used here to capture higher fidelity body movement information. In the weDRAW project, the Kinect is used within an existing platform that includes the EyesWeb XMI body movement analysis package [28][29] [30].
Body movement data has been collected with children while they explored arithmetic concepts in multiple games within the aforementioned scenarios. Analysis of the acquired data (using the EyesWeb XMI package) is ongoing to inform understanding of which cues are expressive in this context. Additional collection of data within games (re-)designed to evoke different levels of selfefficacy and curiosity is underway.
There are three categories of features we are considering in the ongoing movement analysis: low level features (e.g. velocity, energy), spatial/temporal features (e.g. trajectory length, distance covered), motion descriptors (directness, smoothness, impulsivity). These types of features have previously shown efficacy for affect detection [31]. Furthermore, they are related to features that have been used to assess self-efficacy, curiosity, and reflectivity in other contexts. For example, the spatial/temporal features are related to the interaction data found useful in [4] and [6] where, rather than the body and its environment, the interaction medium was a PC and the learning software. As previously discussed, velocity and smoothness have also been found useful in the related study of [25].

CONCLUSION
The aim of this paper was to introduce and foster work in the area of affect-aware learning technology by focusing on three states that are still underexplored despite their importance. To do so, it contributed a brief account of the importance of self-efficacy, curiosity, and reflectivity in learning and the significant impact they have on learning outcomes. We have further reviewed literature to introduce opportunities that technology able to monitor these states can offer for tailoring teaching style and material to enhance the beneficial role of the states and reduce the barriers that low level self-efficacy, lack of curiosity, and cognitive impulsivity may introduce. Finally, we have briefly reviewed literature showing the possibility of automating the monitoring of the states. We have concluded by highlighting bodily cues that may facilitate such monitoring with initial insights on the work in this direction within the weDRAW dataset of children exploring mathematical concepts.