There is a newer version of this record available.

Dataset Open Access

SOTorrent Dataset

Baltes, Sebastian; Dumani, Lorik

Stack Overflow (SO) is the most popular question-and-answer website for software developers, providing a large amount of code snippets and free-form text on a wide variety of topics. Like other software artifacts, questions and answers on SO evolve over time, for example when bugs in code snippets are fixed, code is updated to work with a more recent library version, or text surrounding a code snippet is edited for clarity. To be able to analyze how content on SO evolves, we built SOTorrent, an open dataset based on the official SO data dump. SOTorrent provides access to the version history of SO content at the level of whole posts and individual text or code blocks. It connects SO posts to other platforms by aggregating URLs from text blocks and by collecting references from GitHub files to SO posts. Our vision is that researchers will use SOTorrent to investigate and understand the evolution of SO posts and their relation to other platforms such as GitHub.

The dataset is based on the official Stack Overflow data dump released 2018-03-13 (https://archive.org/details/stackexchange) and the Google BigQuery GitHub data set queried 2018-05-04 (https://cloud.google.com/bigquery/public-data/github). Please read the license files (LICENSE.md) before using the dataset. In the previous release of version 2018-03-28, column PostHistoryTypeId of Table PostVersion was missing.
Files (85.4 GB)
Name Size
1_create_database.sql
md5:50fc1f99b0cd4a1759e2c914e8d527dd
6.5 kB Download
2_load_so_from_xml.sql
md5:6b21ef8943b86a8e10784b774b911dc9
1.2 kB Download
3_create_indices.sql
md5:b2a5e8135b902d7b816e1a15ec9e0442
488 Bytes Download
4_create_sotorrent_tables.sql
md5:9a35f86a014d28e8d49189b0395c0422
6.0 kB Download
5_create_sotorrent_user.sql
md5:4468a089a4174b00fde9c4d385b18149
495 Bytes Download
6_load_sotorrent.sql
md5:a8c3bcb47177b8a37e1b489ce0af2231
4.0 kB Download
7_load_postreferencegh.sql
md5:51692429ac14b6358cdf9e239593fc68
476 Bytes Download
8_create_sotorrent_indices.sql
md5:886fa0e77fbec3c7fbbac94ea8559d66
1.5 kB Download
Badges.xml.gz
md5:ebc9944fd1bed0f43cd07afec6da6d16
277.7 MB Download
Comments.xml.gz
md5:3ab64f1bd41940bdf692a5d282507588
5.5 GB Download
CommentUrl.csv.gz
md5:f5ac85f944c2b7b9ffbdf3364937b08e
269.1 MB Download
LICENSE.md
md5:c861638e5dcbc0426dec06a80f12ea86
38.4 kB Download
PostBlockDiff.csv.gz
md5:ab756dae65f82c52a235abe520227e0a
8.9 GB Download
PostBlockVersion.csv.gz
md5:646728ae54c9a71f3806ce1c3b7b7e40
19.6 GB Download
PostHistory.xml.gz
md5:cc4ca11f111eb17ce43842e145fa9ca0
29.5 GB Download
PostLinks.xml.gz
md5:bc89460a7570723a58c3aa5775b83723
90.6 MB Download
PostReferenceGH.csv.gz
md5:c0afa679ff25bae3986b61614dbb1888
351.5 MB Download
Posts.xml.gz
md5:478708a9cb50c2fd4890e809f02b235d
16.6 GB Download
PostVersion.csv.gz
md5:04a16402361afd53ba497bd0e6e3f0e7
951.1 MB Download
PostVersionUrl.csv.gz
md5:43160bbc8b799ad6657fb905ccb37ff0
886.9 MB Download
README.md
md5:df958adafd3c756cf4c5c9cc8bb51b75
1.4 kB Download
Tags.xml.gz
md5:72deadfbd3f845780ed1fabb8bc7c537
1.0 MB Download
TitleVersion.csv.gz
md5:a7340a82d800162596e19f4e679f32ac
704.8 MB Download
Users.xml.gz
md5:2676ba441dce5beb1f483be8a8af76cc
541.5 MB Download
Votes.xml.gz
md5:fdf1560e480d2bc8471598178ca72784
1.2 GB Download
10,684
32,317
views
downloads
All versions This version
Views 10,684174
Downloads 32,317478
Data volume 249.5 TB1.3 TB
Unique views 7,901170
Unique downloads 6,231112

Share

Cite as