Dataset Open Access

Webis YouTube 8M Augmented 2018

Jiani Qu; Anny Marleen Hißbach; Tim Gollub; Martin Potthast

We used the YouTube Data API to augment the YouTube 8M corpus by crawling a variety of meta data for the videos.

First point of interest was the "video resource," which comprises data about the video, such as the video’s title, description, uploader name, tags, view count, and more. Also included in the meta data is whether comments have been left for the video. If so, we downloaded them as well, including information about their authors, likes, dislikes, and responses.

There is no property which specifies a video’s language, since this information is not mandatory when uploading a video. Also, the API provides only information about the available captions, but not the captions themselves. Only the uploader of a video is given access to its captions via the API; we extracted them using youtube-dl. For each video, all manually created captions were downloaded, and auto-generated captions in the "default" language and English. The "default" auto-generated caption gives perhaps the only hint at a video’s original language.

Finally, we downloaded all thumbnails used to advertise a video, which are not available via the API, but only via a canonical URL. Our corpus provides the possibility to recreate the way a video is presented on YouTube (meta data and thumbnail), what the actual content is ((sub)titles and descriptions), and how its viewers reacted (comments).

If you use this dataset in your publication, please cite the dataset as outlined in the right column.

Files (173.8 GB)
Name Size
captions-list.zip
md5:8dc641a8eda0952b9b9a1f072709c558
995.4 MB Download
captions.zip
md5:8fb3191a31242ff485d49eab091022cd
4.8 GB Download
comments.zip
md5:9c6407509e780eac43b1ba0f4e01c1b2
48.3 GB Download
metadata.zip
md5:62d05a8cea6d772b1288e575c9337e18
7.9 GB Download
thumbnails.zip
md5:37a791ddf259b546f2f9c66ea4924e55
111.9 GB Download
298
258
views
downloads
All versions This version
Views 298298
Downloads 258258
Data volume 19.5 TB19.5 TB
Unique views 268268
Unique downloads 169169

Share

Cite as