Influence of visual cues on head and eye movements during listening tasks in multi-talker audiovisual environments with animated characters

Accepted manuscript, published in Speech Communication (July 2018).

Recent  studies  of  hearing  aid  benefits  indicate  that  head  movement  behavior  influences
performance.  To  systematically  assess  these  effects,  movement  behavior  must  be  measured  in
realistic  communication  conditions.  For  this,  the  use  of  virtual  audiovisual  environments  with
animated  characters  as  visual  stimuli  has  been  proposed.  It  is  unclear,  however,  how  these
animations influence the head- and eye-movement behavior of subjects.  Here, two listening tasks
were carried out with a group of 14 young normal hearing subjects to investigate the influence of
visual cues on head- and eye-movement behavior; on combined localization and speech intelligibility
task  performance;  as well  as  on  perceived  speech intelligibility,  perceived  listening  effort  and  the
general impression of the audiovisual environments. Animated characters with different lip-syncing
and gaze patterns were compared to an audio-only condition and to a video of real persons. Results
show that movement behavior, task performance, and perception were all influenced by visual cues.
The movement behavior of young normal hearing listeners in animation conditions with lip-syncing
was similar to that in the video condition. These results in young normal hearing listeners are a first
step towards using the animated characters to assess the influence of head movement behavior on
hearing aid performance.