Where do the eyes look during "sign-watching"? The impact of early language experience on babies' and children's eye gaze behavior for signed narratives

Where do the eyes look during “sign-watching”? The impact of early language experience on babies’ and children’s eye gaze behavior for signed narratives

Adam Stone and Rain Bosworth

Department of Psychology, University of California, San Diego, USA

Presented at the 3rd International Conference on Sign Language Acquisition, June 2018, Istanbul, Turkey

Questions

What are the eye gaze behaviors of sign-exposed (SE) and non-sign exposed (NSE) babies and children when they are watching signed narratives?  How do gaze behaviors of sign-exposed children change when signed narratives are linguistically disrupted (using video-reversal)?

Background

It is well known that infants are innately sensitive to language patterns in speech. We recently showed that gaze behavior by sign-naïve infants reveal their sensitivity to visual language patterns in isolated signed words (Bosworth et al, 2013) and ASL fingerspelling (Stone, Petitto, & Bosworth, 2017). Also, children’s gaze behavior reflects underlying linguistic knowledge and real-time language processing in both spoken or signed languages (Lieberman et al., 2017; Mani & Huettig, 2014). However, little is known about their gaze behavior while watching natural, signed narratives or how this is affected by innate and experiential factors during development. Here, we examined gaze patterns in SE and NSE babies and children watching ASL narratives. These narratives were presented normally and in reverse (i.e., backwards), which disrupts natural language patterning.

Methods

Using a Tobii X120 eyetracker, we measured gaze behavior during narrative watching in 10 SE and 16 NSE babies (mean ages 0.78, 0.69 years) and in 20 SE and 15 NSE children (mean ages 4.9, 5.0 years). Subjects were presented with 16 short video clips of ASL narratives (8 normal, 8 reversed). We calculated Face-Chest Ratios (FCRs) where the face corresponded to the central eyes, nose, and mouth regions, and the chest to the central neck and torso regions (Figure 1). A positive FCR indicates more face looking relative to chest looking. Linear mixed models were constructed for babies and children separately (outcome variable: FCR; fixed effects: language exposure, video direction, age; random effects: participant, video clip). 

Results

Among babies, we found significant main effects of language experience (t = 2.74, p = 0.011), video reversal (t = 2.08, p = 0.039), and a possible interaction (t = -1.63, p = 0.10). SE babies showed a stronger face preference for normal video compared to reversed video. NSE babies showed chest preferences for both normal and reversed videos. In children, we found only significant effects of language experience (t = 2.26, p = 0.030) where SE children strongly preferred to watch the face but NSE children showed a slight chest preference (Figure 2).

Discussion

The findings suggest language experience affects eye gaze early in life (~6-9 months old) and interact with infants’ pre-existing sensitivity to language patterning, as evidenced by both SE and NSE babies’ responses to reversed video. We hypothesize that SE babies quickly adopt gaze behaviors well-suited to sign language comprehension, but these behaviors are affected by linguistic disruption (reversed video). By 5 years old, SE children have adopted resilient, face-focused gaze behaviors similar to those of deaf SE adults (Bosworth, Hwang, & Stone, in prep). NSE babies and children attend to visually salient parts of the scene, including the moving hands. The results argue for the critical importance of early visual language exposure for deaf and hard of hearing infants.