OntoLAMA: LAnguage Model Analysis for Ontology Subsumption Inference

About

OntoLAMA is a set of language model (LM) probing datasets for ontology subsumption inference. The work follows the "LMs-as-KBs" literature but focuses on conceptualised knowledge extracted from formalised KBs such as the OWL ontologies. Specifically, the subsumption inference (SI) task is introduced and formulated in the Natural Language Inference (NLI) style, where the sub-concept and the super-concept involved in a subsumption axiom are verbalised and fitted into a template to form the premise and hypothesis, respectively. The sampled axioms are verified through ontology reasoning. The SI task is further divided into Atomic SI and Complex SI where the former involves only atomic named concepts and the latter involves both atomic and complex concepts. Real-world ontologies of different scales and domains are used for constructing OntoLAMA and in total there are four Atomic SI datasets and two Complex SI datasets.

Citation

The relevant paper has been accepted at Findings of ACL 2023: https://aclanthology.org/2023.findings-acl.213/.

```
@inproceedings{he-etal-2023-language,
    title = "Language Model Analysis for Ontology Subsumption Inference",
    author = "He, Yuan  and
    Chen, Jiaoyan  and
    Jimenez-Ruiz, Ernesto  and
    Dong, Hang  and
    Horrocks, Ian",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.findings-acl.213",
    doi = "10.18653/v1/2023.findings-acl.213",
    pages = "3439--3453"
}
```

Links

• See instructions at: https://krr-oxford.github.io/DeepOnto/ontolama/
• We have made available a convenient access of these datasets through Huggingface: https://huggingface.co/datasets/krr-oxford/OntoLAMA
• The arxiv version is available at: https://arxiv.org/abs/2302.06761