FluentSigners-50: a signer independent benchmark dataset for Sign Language Processing

A new large-scale Kazakh-Russian Sign Language dataset (FluentSigners-50) as a new Continuous Sign Language Recognition benchmark.  FluentSigners-50 proposes to address three shortcomings of commonly used datasets: continuous signing, signer variety, and native signers. FluentSigners-50's main advantage is in its large signer variety: age (ranging from 8 to 57 years old), gender (18 male and 32 female), clothing, skin tone, body proportions, disability (deaf or hard of hearing), and fluency. Additionally, as the dataset was crowd-sourced: the participants were using a variety of their own recording devices (such as smartphones and web cameras), it resulted in a large variety of backgrounds, lighting conditions, camera quality, frame rates, camera aspect ratios, and angles. Finally, FluentSigners-50 contains recordings of 50 contributors that use sign language on a daily basis: either deaf, hard of hearing, hearing CODA (Child of Deaf Adults), and hearing SODA (Sibling of a Deaf Adult). As a result, the dataset contains a high degree of linguistic variability, including phonetic, phonological, lexical, and syntactic variations. It thus is a better training set for recognition of natural signing.

The FluentSigners-50 dataset consists of everyday conversational phrases and sentences in KRSL, the sign language used in the Republic of Kazakhstan. KRSL is closely related to Russian Sign Language (RSL) and some other sign languages of the ex-Soviet Union. While no official research comparing KRSL with RSL exists, our observations based on our experience researching both languages are that they show a substantial lexical overlap and are entirely mutually intelligible. The sentences and phrases of FluentSigners-50 represent the following sentence types: statements, polar questions, wh-questions, and requests.

All FluentSigners-50 contributors use sign language on a daily basis as they are either deaf (N=32), hard of hearing (N=6), hearing SODA (N=3), or hearing CODA (N=9). Native signers are signers who have been exposed to signed languages since birth because their parents are deaf. While the early acquisition may be necessary for the development of native language abilities, other factors, particularly the quality of language input, may play a role. According to this distinction, FluentSigners-50 has 30 CODA contributors (including nine hearing signers) and 20 who are not CODA (16 deaf, one hard of hearing, and three hearing SODA). Nevertheless, we decided to name our dataset FluentSigners-50 because all of our contributors use sign language daily, and it is their primary language of communication. They all came from various regions of Kazakhstan and are of different age and gender groups.