GitTables 1M

Note: version 0.0.5 includes the files of version 0.0.4, yielding duplication in topic subsets. This will be fixed soon (ETA: 5 May 2022).

Summary

GitTables (https://gittables.github.io) is a corpus of currently 1.7M relational tables extracted from CSV files on GitHub. We aim to grow this to at least 10M tables. Each file in this corpus represents a table with the original content (e.g. values and header) as extracted from the corresponding CSV file. Table columns were also annotated with >2K semantic types from Schema.org and DBpedia (provided separately). These column annotations consist of, for example, semantic types, hierarchical relations to other types, and descriptions.

We believe GitTables can facilitate many use-cases, among which:

• Data integration, search and validation.

• Data visualization and analysis recommendation.

• Schema analysis and completion for e.g. database or knowledge base design.

If you have questions, the paper, documentation, and contact details are provided on the website: https://gittables.github.io. We recommend using Zenodo's API to download the full dataset (i.e. all zipped topic subsets).

Dataset contents

The data is provided in subsets of tables stored in parquet files, each subset corresponds to a term that was used to query GitHub with. The column annotations and other metadata (e.g. URL) are attached to the metadata of the parquet file. This version corresponds to this version of the paper https://arxiv.org/abs/2106.07258v4.

In summary, this dataset can be characterized as follows:

Future releases

Future releases will include the following:

• Increased number of tables (expected at least 10M)