Automatically Predicting Peer Satisfaction During Collaborative Learning with Linguistic, Acoustic, and Visual Features

Collaborative learning has numerous benefits such as enhancing learners’ critical thinking, developing
social skills, and improving learning gains. While engaging in this interactive process, learners’ satisfaction
toward their partners plays a crucial role in defining the success of the collaboration. However,
detecting learners’ satisfaction during an ongoing collaboration remains challenging, and there are no
automatic techniques to predict learners’ satisfaction. In this paper, we propose a multimodal approach
to automatically predict peer satisfaction for co-located collaboration with features extracted from 44
middle school learners’ collaborative dialogues. We investigated three types of features extracted from
learners’ dialogues: 1) linguistic features indicating semantics and sentiment; 2) acoustic-prosodic features
including energy and pitch; and 3) visual features including eye gaze, head pose, facial action units,
and body pose. We then trained several regression models with each of those features to predict the peer
satisfaction scores that learners received from their partners. The results revealed that head position and
body location were significant indicators of peer satisfaction: lower head and body distances between
partners were associated with more positive peer satisfaction. Next, we investigated the influence of multimodal
feature fusion methods on peer satisfaction prediction accuracy: early fusion versus late fusion.
We report the comparison results between models trained with (1) best-performing unimodal features, (2)
multimodal features combined by early fusion, and (3) multimodal features combined by late fusion. This
line of research reveals how multimodal features from collaborative dialogues are associated with peer
satisfaction, and represents a step toward the development of real-time intelligent systems that support
collaborative learning.