Human-centric Evaluation of Semantic Resources: A Systematic Mapping Study

As ontologies and knowledge graphs increasingly power intelligent applications across diverse domains, ensuring their quality remains a critical challenge. While automated tools are useful, they often fail to capture quality criteria, for example, such as domain and modeling correctness, which require the nuanced assessment only human participation can provide. Despite the essential role of such human-centric evaluation of semantic resources (HESR), the field has lacked a unified theoretical framework and a clear overview of best practices. In this talk, we address this gap by introducing a new perspective that integrates existing literature into a comprehensive theoretical framework for defining and characterizing HESR. Furthermore, we will also present the results of a Systematic Mapping Study covering 144 papers published over the last 15 years, empirically grounding our framework and identifying key trends in this area.