Empirical Software Engineering
Published February 9, 2017 | Version v1
Dataset Open

Programming language keyword frequencies extracted from 16,000,000 public GitHub repositories (October 2016)

Creators

  • 1. source{d}

Description

Origin

16,000,000 repositories on GitHub as of October 2016, classified with github/linguist and parsed with Pygments. Token.Keyword tokens were filtered and MapReduce-d. Fuzzy duplicate repositories were discarded.

Some languages, e.g. Haskell, are parsed wrong, resulting in many keywords. Still they were not removed since we are not familiar with such languages.

Format

Triples [language name]\t[keyword]\t[frequency]

Tabs and new lines in keywords are escaped as \t and \n respectively.

Files

Files (47.4 MB)

Name Size Download all
md5:0202530008708c280cc7e641f6754596
47.4 MB Download

Additional details

Related works

Is identical to
https://data.world/vmarkovtsev/github-lng-keyword-frequencies (URL)