There is a newer version of this record available.

Dataset Open Access

SOTorrent Dataset

Baltes, Sebastian; Dumani, Lorik

Stack Overflow (SO) is the most popular question-and-answer website for software developers, providing a large amount of code snippets and free-form text on a wide variety of topics. Like other software artifacts, questions and answers on SO evolve over time, for example when bugs in code snippets are fixed, code is updated to work with a more recent library version, or text surrounding a code snippet is edited for clarity. To be able to analyze how content on SO evolves, we built SOTorrent, an open dataset based on the official SO data dump. SOTorrent provides access to the version history of SO content at the level of whole posts and individual text or code blocks. It connects SO posts to other platforms by aggregating URLs from text blocks and comments, and by collecting references from GitHub files to SO posts. Our vision is that researchers will use SOTorrent to investigate and understand the evolution of SO posts and their relation to other platforms such as GitHub.

If you use this dataset in your work, please cite our MSR 2018 paper (BibTex) or our MSR 2019 mining challenge proposal.

The dataset is based on the official Stack Overflow data dump released 2019-03-04 (https://archive.org/details/stackexchange) and the Google BigQuery GitHub data set queried 2019-03-29 and last updated 2019-03-29 according to table info (https://cloud.google.com/bigquery/public-data/github). Please read the license files (LICENSE.md) before using the dataset.
Files (64.3 GB)
Name Size
1_create_database.sql
md5:b4405ab4a5bde96c0b425b0309b5f6dc
6.5 kB Download
2_create_sotorrent_user.sql
md5:9c7851cdadeee18ab5559826bc81e1f4
352 Bytes Download
3_load_so_from_xml.sql
md5:f272f5f6aee0af40d2ee21995a04ab19
1.2 kB Download
4_create_indices.sql
md5:e3d316bab798120f31102ca843c9a869
488 Bytes Download
5_create_sotorrent_tables.sql
md5:254d10edd65593d4ee24b4b76a476a9e
6.8 kB Download
6_load_sotorrent.sql
md5:0cb78fbcd5dae47f05969f60e4d5b83c
4.6 kB Download
7_load_postreferencegh.sql
md5:ed605e0518dbe6f87ed35dfae57a33d3
535 Bytes Download
8_load_ghmatches.sql
md5:4de9746a8629f3d4d699f4cf65ce64ad
469 Bytes Download
9_create_sotorrent_indices.sql
md5:d65f16daa9a4257f69da78d6e2f8151a
1.6 kB Download
Badges.xml.7z
md5:2a12daa2eff8e853bd2dbcbd105d3769
246.9 MB Download
Comments.xml.7z
md5:cd7638a8b8d68163c5f7b5088d816963
4.3 GB Download
CommentUrl.csv.7z
md5:7ff076ce76dc3bddb9decdc248baf406
303.7 MB Download
GHMatches.csv.7z
md5:3abf1df287699a71b6f0a60b79adf1ec
841.6 MB Download
LICENSE.md
md5:dd690a9239427df0ccf2369c97285125
38.4 kB Download
PostBlockDiff.csv.7z
md5:6b99e217c397ac5521b41216c1b4b2a2
7.2 GB Download
PostBlockVersion.csv.7z
md5:dd34e1ca6d41b0701d4f27fcd3670bf1
15.5 GB Download
PostHistory.xml.7z
md5:03f6aa6ab6c7ea1c3899f159b4c3d9ea
18.7 GB Download
PostLinks.xml.7z
md5:09d5377f930e5f908f2648f3fff58f3d
78.9 MB Download
PostReferenceGH.csv.7z
md5:a8bdc2595a0ac9e1c91cc26beb70f727
172.1 MB Download
Posts.xml.7z
md5:04e7509a14b05532b037eef601064039
13.1 GB Download
PostVersion.csv.7z
md5:45db49348f855edef6ebd8d669ace6e3
723.3 MB Download
PostVersionUrl.csv.7z
md5:dae5d04e15a6fea5ae46db9cd4cbb9f9
1.1 GB Download
README.md
md5:a2119be9b9d438fd650e3cc9b8a73190
3.1 kB Download
Tags.xml.7z
md5:8be9ecece96f4617c61acedae55b5517
754.2 kB Download
TitleVersion.csv.7z
md5:ae7acb28d9d56d9735c1348d33e11e1f
531.5 MB Download
Users.xml.7z
md5:edc7ff89c964d7ffdffbb28639910e97
507.2 MB Download
Votes.xml.7z
md5:dc595a4a9f965e08db7278bced14170a
1.1 GB Download
3,288
10,664
views
downloads
All versions This version
Views 3,288287
Downloads 10,664504
Data volume 58.1 TB1.8 TB
Unique views 2,638235
Unique downloads 2,007124

Share

Cite as