Dataset Open Access

SOTorrent Dataset

Baltes, Sebastian; Dumani, Lorik

Stack Overflow (SO) is the most popular question-and-answer website for software developers, providing a large amount of code snippets and free-form text on a wide variety of topics. Like other software artifacts, questions and answers on SO evolve over time, for example when bugs in code snippets are fixed, code is updated to work with a more recent library version, or text surrounding a code snippet is edited for clarity. To be able to analyze how content on SO evolves, we built SOTorrent, an open dataset based on the official SO data dump. SOTorrent provides access to the version history of SO content at the level of whole posts and individual text or code blocks. It connects SO posts to other platforms by aggregating URLs from text blocks and comments, and by collecting references from GitHub files to SO posts. Our vision is that researchers will use SOTorrent to investigate and understand the evolution of SO posts and their relation to other platforms such as GitHub.

If you use this dataset in your work, please cite our MSR 2018 paper (BibTex) or our MSR 2019 mining challenge proposal.

The dataset is based on the official Stack Overflow data dump released 2019-09-04 (https://archive.org/details/stackexchange) and the Google BigQuery GitHub data set queried 2019-09-23 and last updated 2019-09-20 according to table info (https://cloud.google.com/bigquery/public-data/github). Please read the license files (LICENSE.md) before using the dataset.
Files (68.0 GB)
Name Size
Badges.xml.7z
md5:eaac4d06794c477967d66f5504005b1c
264.0 MB Download
Comments.xml.7z
md5:ed8edfdde9030948b9b67cf81283d387
4.5 GB Download
CommentUrl.csv.7z
md5:9502a99359133c07b5588a200efdea1f
321.6 MB Download
GHMatches.csv.7z
md5:1adfed220a3470bfbda85751e04818ab
838.2 MB Download
LICENSE.md
md5:9b09d402f3eb0c135df5c2e27dbc1cc4
16.6 kB Download
load_sotorrent.sh
md5:9c66d07fc465b34732198f9b147ab9d8
2.7 kB Download
PostBlockDiff.csv.7z
md5:bcd6b1a89aa4fa17cbff60aa8960177d
7.7 GB Download
PostBlockVersion.csv.7z
md5:5ac41482fb4bd1850e74b477efade459
16.3 GB Download
PostHistory.xml.7z
md5:b67209c7eca019befffd7d5833a03c73
19.7 GB Download
PostLinks.xml.7z
md5:43cf744330da08dbcbb6a4876f75a43b
84.3 MB Download
PostReferenceGH.csv.7z
md5:9b21406809d054ceb03f7baa56ecd12a
172.1 MB Download
Posts.xml.7z
md5:56001771f3b18741feabf6a974f0eda7
13.8 GB Download
PostVersion.csv.7z
md5:32e0e6aa33f994437bc46861465d35a7
759.6 MB Download
PostVersionUrl.csv.7z
md5:fadd8ae4371104cc716b250ee4657b8e
1.1 GB Download
README.md
md5:e75068fdaf3f634d10552bcdc871ed20
2.4 kB Download
sql.7z
md5:5901639230389ffa46086775b13f7345
4.1 kB Download
StackSnippetVersion.csv.7z
md5:b1357c112d554a3adef3a668ef7aded6
235.5 MB Download
Tags.xml.7z
md5:20de42eeda8055dba6793812b9394990
772.7 kB Download
TitleVersion.csv.7z
md5:bacf9c029879c9b0cb2f1293c93a1271
565.2 MB Download
Users.xml.7z
md5:33d236dd3d3babedf2a3ffe7c25ede10
559.6 MB Download
Votes.xml.7z
md5:fdef8800cf65653cc2e818495e6c638f
1.2 GB Download
3,497
13,629
views
downloads
All versions This version
Views 3,49766
Downloads 13,6292,831
Data volume 84.4 TB25.3 TB
Unique views 2,78955
Unique downloads 2,09956

Share

Cite as