There is a newer version of this record available.

Dataset Open Access

SOTorrent Dataset

Baltes, Sebastian; Dumani, Lorik

Stack Overflow (SO) is the most popular question-and-answer website for software developers, providing a large amount of code snippets and free-form text on a wide variety of topics. Like other software artifacts, questions and answers on SO evolve over time, for example when bugs in code snippets are fixed, code is updated to work with a more recent library version, or text surrounding a code snippet is edited for clarity. To be able to analyze how content on SO evolves, we built SOTorrent, an open dataset based on the official SO data dump. SOTorrent provides access to the version history of SO content at the level of whole posts and individual text or code blocks. It connects SO posts to other platforms by aggregating URLs from text blocks and comments, and by collecting references from GitHub files to SO posts. Our vision is that researchers will use SOTorrent to investigate and understand the evolution of SO posts and their relation to other platforms such as GitHub.

If you use this dataset in your work, please cite our MSR 2018 paper (BibTex) or our MSR 2019 mining challenge proposal.

The dataset is based on the official Stack Overflow data dump released 2018-06-05 (https://archive.org/details/stackexchange) and the Google BigQuery GitHub data set queried 2018-08-28 (https://cloud.google.com/bigquery/public-data/github). Please read the license files (LICENSE.md) before using the dataset.
Files (88.6 GB)
Name Size
1_create_database.sql
md5:2c884318d9344c6c4d5bb0815a0d7bd6
6.7 kB Download
2_load_so_from_xml.sql
md5:799f3fd4fde2522a8acfbf10874b2332
1.2 kB Download
3_create_indices.sql
md5:0ed7536b5ab713c5f77ae5699604c608
501 Bytes Download
4_create_sotorrent_tables.sql
md5:a86775925090ad026e64d0e903bc8de4
6.6 kB Download
5_create_sotorrent_user.sql
md5:3f1228feae388d0c8c35570f39ee6300
506 Bytes Download
6_load_sotorrent.sql
md5:33271fc91ff9532172fbc29677ea5625
4.7 kB Download
7_load_postreferencegh.sql
md5:73693f9ef65547c799aed4d0d98ba474
489 Bytes Download
8_create_sotorrent_indices.sql
md5:cfb179350388e38133c83bbf802c1ce2
1.5 kB Download
Badges.xml.gz
md5:fdf30e1e366f1d150337c848f55b8492
289.1 MB Download
Comments.xml.gz
md5:686f3c9ec9d4dee3d19893a5ef96303d
5.6 GB Download
CommentUrl.csv.gz
md5:bc2bf8a54d96316d4e461fffa7315b41
401.7 MB Download
LICENSE.md
md5:f02ef134b20bfe264755fb4338f47ccf
39.1 kB Download
PostBlockDiff.csv.gz
md5:3b0b47ad37b89faf734e1ff55a6f59a0
9.2 GB Download
PostBlockVersion.csv.gz
md5:3717c064609812ffaf4591b2e95360d1
20.3 GB Download
PostHistory.xml.gz
md5:b541fff7b494ef8c56bad459b2bf9d5e
30.4 GB Download
PostLinks.xml.gz
md5:b5918b15a5f400217300d9d60b7425bd
94.3 MB Download
PostReferenceGH.csv.gz
md5:bdcd92fa18be6c47ef7a4239585ff606
355.0 MB Download
Posts.xml.gz
md5:b8620f67212361096743fa6d65fb40ac
17.1 GB Download
PostVersion.csv.gz
md5:002266ea35e5fdd8728187e302643570
979.0 MB Download
PostVersionUrl.csv.gz
md5:e4a9f3675b7912ffa80b4cdd3a57193f
1.3 GB Download
README.md
md5:f5d184bb8fc9931eac19863c0dcc42f0
1.4 kB Download
Tags.xml.gz
md5:0f456720abf859ca4d9cdc5965d78685
1.0 MB Download
TitleVersion.csv.gz
md5:271e50be47bba231b8a7fd4afb74db22
726.7 MB Download
Users.xml.gz
md5:c9a8969fbf4a7ea42e6c2135860f53de
567.5 MB Download
Votes.xml.gz
md5:c821ce150972429ef2d247d2e2df9d11
1.3 GB Download
1,198
4,746
views
downloads
All versions This version
Views 1,198143
Downloads 4,746449
Data volume 20.8 TB2.1 TB
Unique views 967132
Unique downloads 75070

Share

Cite as