CoDaKG: Content-Based Dataset Knowledge Graphs for Dataset Search

Description

This repository contains the dataset resources for the paper "CoDaKG: Content-Based Dataset Knowledge Graphs for Dataset Search". It provides two Content-based Dataset Knowledge Graph (CoDaKG) instances constructed for the NTCIR and ACORDAR test collections, along with the underlying pairwise similarity data derived from file content analysis.

File Inventory

1. Knowledge Graph Dumps (RDF/Turtle) 

These files contain the full CoDaKG instances in .ttl format. 

• ntcir_full_rdflib.ttl: The CoDaKG instance constructed for the NTCIR test collection.

• acordar_full_rdflib.ttl: The CoDaKG instance constructed for the ACORDAR test collection.

2. Content Similarity Data (CSV)

The following CSV files contain the raw pairwise similarity scores calculated between dataset distributions. These files only include pairs with a Jaccard similarity threshold > 0.5.

Files:

• NTCIR_dataOverlap_similarity.csv

• NTCIR_schemaOverlap_similarity.csv

• ACORDAR_dataOverlap_similarity.csv

• ACORDAR_schemaOverlap_similarity.csv

Column Structure:

• file_id1 / file_id2: The identifier for the dcat:Distribution (file).

• dataset_id1 / dataset_id2: The identifier for the corresponding dcat:Dataset.

• sim: The calculated Jaccard similarity score (float).

Similarity Types:

• Schema Overlap: Similarity based on structural elements (e.g., column headers, JSON paths).

• Data Overlap: Similarity based on data content units (e.g., table rows, sentence overlaps).

Usage Notes:

• ID Correspondence: The file_id and dataset_id in the CSV files correspond to the local names of the URIs for dcat:Distribution and dcat:Dataset entities in the Turtle (.ttl) files.

• Customizing Thresholds: To revise the Knowledge Graph using different similarity thresholds:

  1. Remove all existing triples with the predicates base:schemaOverlap and base:dataOverlap from the .ttl file.

  2. Filter the CSV files to select pairs meeting your desired threshold.

  3. Generate and append new triples linking the corresponding entities based on their IDs.