Human-centric Evaluation of Semantic Resources:A Systematic Mapping Study Protocol and Data

We perform a Systematic Mapping Study on the topic of Human-centric Evaluation of Semantic Resources. 

In this resource, we include:

• the study protocol we followed to conduct the study
  • HESR_SMS_Protocol.pdf

• the collected data from 100 primary studies (as a google sheet/csv/knowledge graph)
  • HESR_SMS_ExtractedData Google Sheet 
  • HESR_SMS_ExtractedData.csv
  • HESR_SMS_ExtractedData_ColumValues.csv
  • HESR_KG.owl

• the implemented data analysis notebooks
  • Bibliographic Analysis (Google Colaboratory Notebooks)
    • D2_D3_PapersPerYear_PaperTypes.ipynb 
    • D5_AuthorAffiliations.ipynb
    • D6_Keywords.ipynb
    • D23_D28_PaperQualty.ipynb
  • RQ1. Semantic Resource Analysis (Google Colaboratory Notebooks)
    • D7a_D7b_ResourceTypes_AuthorVsReviewer.ipynb
    • D7b_ResourceTypesReviewer.ipynb
    • D7c_EvaluatedResources(PerDomain).ipynb
    • D8_SizeOfResource.ipynb
    • D9_FormalismOfResource.ipynb
  • RQ2. Evlaution Process Analysis (Google Colaboratory Notebooks)
    • D10_VerifiedAspect.ipynb
    • D11_FrameOfReference.ipynb
    • D12_ContextOfEvaluation.ipynb
    • D13_EvaluationMethod.ipynb
    • D14_EvaluationModality.ipynb
    • D15_BiasAddressed.ipynb
  • RQ3. Evaluation Population Analysis (Google Colaboratory Notebooks)
    • D16_EvaluatorTask.ipynb
    • D17_D22_PopulationCharacteristics.ipynb