Published April 7, 2018 | Version v1
Preprint Open

Using Shape Expressions (ShEx) to Share RDF Data Models and to Guide Curation with Rigorous Validation

  • 1. Yale University
  • 2. Mayo Clinic
  • 3. The Scripps Research Institute
  • 4. University of Oviedo
  • 5. Data Science Institute, University of Virginia
  • 6. World Wide Web Consortium
  • 7. Micelio

Description

We discuss Shape Expressions (ShEx), a concise, formal, modeling and validation language for RDF structures.
For instance, a Shape Expression
could prescribe
that subjects in a given RDF graph that fall into the shape ``Paper'' are expected to have a section called ``Abstract'', and any ShEx implementation can confirm whether that is indeed the case for all such subjects within a given graph or subgraph.

There are currently five actively maintained ShEx implementations. We discuss how we use the Javascript, Scala and Python implementations in RDF data validation workflows in distinct applied contexts. We present examples of how ShEx can be used to model and validate data from two different sources---the domain-specific Fast Healthcare Interoperability Resources (FHIR) and the domain-generic Wikidata knowledge base, which is the linked database built and maintained by the Wikimedia Foundation as a sister project to Wikipedia. Three projects that are using Wikidata as a data curation platform are presented as well, along with ways in which they are using ShEx for modeling and validation.

When reusing RDF graphs created by others, it is important to know how the data is represented. Current practices of using human-readable descriptions or ontologies to communicate data structures often lack sufficient precision for
data consumers
to quickly and easily understand data representation details. We provide concrete examples of how we use ShEx as a constraint and validation language that allows humans and machines to communicate unambiguously about data assets.
We use ShEx to exchange and understand data models of different origins, and to express a shared model of a resource's footprint
in a linked data source. We also use ShEx to agilely develop data models, test them against sample data, and revise or refine them. The expressivity of ShEx allows to catch disagreement, inconsistencies, or errors efficiently both at the time of input, and through batch inspections.

ShEx addresses the need of the semantic web community to ensure data quality for RDF graphs. It is currently being used in the development of FHIR/RDF.  
The language is sufficiently expressive to capture constraints in FHIR, and the intuitive syntax helps people to quickly grasp the range of conformant documents. The publication workflow for FHIR tests all of these examples against the ShEx schemas, catching non-conformant data before they reach the public.
ShEx is also currently used in Wikidata projects such as Gene Wiki and WikiCite to develop quality-control pipelines to maintain data integrity and incorporate or harmonize differences in data across different parts of the pipelines. We end by discussing how ShEx validation of Wikidata subgraphs about software and file formats enables new approaches to digital preservation of software and data, including Emulation as a Service.

Notes

Preprint submitted to the In-Use track of the 17th International Semantic Web Conference (ISWC 2018).

Files

shape-expressions-shex.pdf

Files (202.5 kB)

Name Size Download all
md5:57bc5a106c56c9aca6d77746c12fbd5a
202.5 kB Preview Download