GENEA Challenge 2023 user-study video stimuli

This Zenodo repository contains video stimuli in mp4 format from the user studies in the GENEA Challenge 2023.

Contents:

The file "monadic_videos.zip" contains the video stimuli used in the monadic evaluations of the challenge (human-likeness and speech appropriateness) and the file "dyadic_videos.zip" the video stimuli from the dyadic evaluation (appropriateness for the interlocutor).

Video stimuli with mismatched motion are in the corresponding subfolders with the prefix "mismat_".

The file "attention_check_examples.zip" contains examples of the attention-check videos used in the challenge.

The release contains all videos used for the challenge evaluation, except for the attention-checks, where only a few examples are provided. Except for the attention-check examples for the human-likeness studies, all videos contain speech audio. This audio needs to be removed to replicate the human-likeness evaluation.

Attribution:

If you use this material, please cite our latest paper on the GENEA Challenge 2023. At the time of writing (2023-08-01) this is our ACM ICMI 2023 paper:

Taras Kucherenko, Rajmund Nagy, Youngwoo Yoon, Jieyeon Woo, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2023. The GENEA Challenge 2023: A large-scale evaluation of gesture generation models in monadic and dyadic settings. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI ’23). ACM.

Also, please cite the paper about the original dataset from Meta Research:

Gilwoo Lee, Zhiwei Deng, Shugao Ma, Takaaki Shiratori, Siddhartha S. Srinivasa, and Yaser Sheikh. 2019. Talking With Hands 16.2M: A large-scale dataset of synchronized body-finger motion and audio for conversational motion analysis and synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV ’19). IEEE, 763–772.

Condition NA in the data contains motion from the Talking With Hands 16.2M dataset at https://github.com/facebookresearch/TalkingWithHands32M/. These stimuli, and the audio, are licensed under a CC BY NC 4.0 international license. The motion for all other conditions is released under a CC BY 4.0 international license, whose license text is reproduced in the file LICENSE.txt.

More information:

To find more GENEA Challenge 2023 material on the web, please see:

• https://genea-workshop.github.io/2023/challenge/

If you have any questions or comments, please contact:

• The GENEA Challenge organisers <genea-challenge@googlegroups.com>