WikiSnap25: Datasets for Wikipedia Article and Wikidata Entity Research

WikiSnap25 is a collection of analysis‑ready tables that integrate content, metadata, and readership information for English Wikipedia articles and Wikidata entities as of mid‑2025. The goal is to lower the engineering barrier for empirical Wikimedia research by abstracting away the work of parsing XML/JSON dumps and aggregating pageview data into compact, well‑documented Parquet datasets.

WikiSnap25 combines several official Wikimedia sources: the June 1, 2025 English Wikipedia pages‑articles‑multistream XML dump for article text and metadata, the June 2, 2025 full Wikidata JSON dump for entity‑level information, a decade of Wikimedia REST API monthly pageview data from July 2015 through June 2025, and additional dumps (e.g., stub meta‑history and redirect mappings) for revision context and canonicalization. A modular processing pipeline (XML parsing, wikitext processing, mwparserfromhell, and aggregation scripts) produces a consistent article‑level snapshot anchored at mid‑2025, enriched with long‑term readership and knowledge‑graph features.

The following datasets are included:

• WikiSnap_2025_WP_Articles.parquet – Article‑level data for 7,011,415 English Wikipedia articles, including normalized titles, total pageviews (2015–2025), first two sentences of article text, outlink metrics (e.g., num_articles_linked_out), and mappings to Wikidata QIDs. This table supports studies of article popularity, connectivity, and entity types.

• WikiSnap_2025_WP_Edges.parquet – 314,337,621 intra‑Wikipedia hyperlinks between articles, with from/to article IDs and titles, endpoint pageviews, and a duplicate‑link flag for graph cleaning. This edge list is suitable for large‑scale network analyses of the article link graph.

• WikiSnap_2025_WP_Network_Metrics.parquet – Network metrics for articles with intra-links to other Wikipedia articles (7,008,375 rows), including centrality scores, PageRank, traffic‑weighted PageRank, HITS hub/authority, degrees, and Leiden community assignments computed from a deduplicated article link graph.

• WikiSnap_2025_WP_Article_Metrics.parquet – Additional article‑level metrics (7,011,415 rows) such as creation timestamp, edit lifespan in days, total revisions, number of unique editors, and a Gini‑style editor inequality index across redirect clusters.

• WikiSnap_2025_WD_Entities.parquet – Wikidata entity information for 116,183,072 items, including labels, instance‑of labels, temporal attributes (begin/end year), country of citizenship labels, and sitelinks to English Wikipedia article titles, with disambiguation pages filtered and truthy claims prioritized.

• WikiSnap_2025_WD_Metrics.parquet – Wikidata metrics for the same 116M entities, including counts of claims, in/out links, references, qualifiers, notable properties, article‑quality badges, and external identifiers, enabling notability and completeness assessments.

• WikiSnap_2025_WD_Humans.parquet – A subset for human entities (P31=Q5) with 12,371,744 rows, containing birth and death years, occupation labels, gender, citizenship, and notability metrics, tailored to demographic and biographical analysis.

These datasets are intended to support a wide range of tasks, including correlating network centrality with pageview patterns, assessing Wikidata notability via property richness and sitelinks, and exploring editorial activity and inequality, without requiring researchers to build their own large‑scale Wikimedia processing pipelines from scratch.

Limitations. The current release focuses on English Wikipedia main‑namespace articles as of June 1, 2025; Wikidata content as of early June 2025; and article‑level pageviews aggregated over July 2015–June 2025 without finer temporal granularity. Redirects, stubs, and full revision histories are not included in these tables.

All datasets and the full processing code are released to facilitate reproducible Wikimedia research and to enable cross‑study comparability on a shared, 2025‑anchored snapshot.

License and citation. All WikiSnap25 datasets and associated processing code archived in this record are dedicated to the public domain under the Creative Commons Zero v1.0 Universal (CC0 1.0) public domain dedication. Users are free to reuse, modify, and redistribute the materials without restriction. When using WikiSnap25 in academic or applied work, please cite this record’s DOI and include a reference to the WikiSnap25 paper abstract where appropriate.