PyKEEN Benchmarking Experiment Model Files

This repository provides weights of the models from the benchmarking study conducted in "Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework" which have been upgraded to compatible with PyKEEN 1.9.

The weights are organized as zipfiles, which are named by the dataset-interaction function configuration. For each of these combinations, we chose the best according to validation Hits@10 to include into this repository. For each model, we have three files:

1. configuration.json contains the (pipeline) configuration used to train the model. It can loaded as

import pathlib
import json
configuration = json.loads(pathlib.Path("configuration.json").read_text())

Since the configuration is intended for the pipeline, we need some custom code to re-create the model without re-training it.

from pykeen.datasets import get_dataset
from pykeen.models import ERModel, model_resolver

configuration = configuration["pipeline"]
# load the triples factory
dataset = get_dataset(
    dataset=configuration["dataset"], dataset_kwargs=configuration.get("dataset_kwargs", None)
)
model: ERModel = model_resolver.make(
    configuration["model"], configuration["model_kwargs"], triples_factory=dataset.training
)

Note, that this only creates the model instance, but does not load the weights, yet.

1. state_dict.pt contains the weights, stored via torch.save. They can be loaded via

import torch
state_dict = torch.load("state_dict.pt")

We can load these weights into the model by using Module.load_state_dict

model.load_state_dict(state_dict, strict=False)

Note that we set strict=False, since the exported weights do not contain regularizers' state, while the re-instantiated models may have regularizers.

1. results.json contains the results obtained by the original runs. It can be read by

import pathlib
import json
configuration = json.loads(pathlib.Path("results.json").read_text())

Note that some of the recently added metrics are not available in those results.