Data set of "Context-sensitive Requirements Search in Natural Language Specifications"

We use this data set for the evaluation of our Context-sensitive Requirements Search approach. It contains three files and 15 searches.

Attribution

The foundation of this data set is PURE:
A. Ferrari, G. O. Spagnolo, and S. Gnesi. PURE: a Dataset of Public Requirements Documents. Version 1.0. Sept. 2018.
Available at: https://doi.org/10.5281/zenodo.1414117

The original data is licensed under Creative Commons Attribution 4.0 International

Annotations

• Each of the provided files contains annotations that we created with Label Studio for the corresponding file of the PURE data set.
• These annotations depict the search results of Plain Keyword Search (PKS) and the corresponding gold standard.
• The following is an example of an annotation in the .json file:

{
    "id": "UnaxUjB6oY",
    "meta": { "text": ["0"] },
    "type": "labels",
    "value": {
        "end": "/text()[2]",
        "text": "speed",
        "start": "/text()[2]",
        "labels": ["speed: train speed profile"],
        "endOffset": 364,
        "startOffset": 359,
        "globalOffsets": { "end": 1403, "start": 1398 }
    },
    "origin": "manual",
    "to_name": "text",
    "from_name": "ner"
},

Important Json attributes to consider

Meta

We encode the relevance of each result in its metadata.

Example:

"meta": { "text": ["0"] },

• 1 = relevant for this search and found by PKS
• 0 = irrelevant for this search but found by PKS
• -1 = relevant for this search and not found by PKS

Labels

This element is the search identifier we use in Label Studio. It corresponds to the table in the thesis that describes each search in detail.

Example:

"labels": ["speed: train speed profile"],

GlobalOffsets

"globalOffsets" describes the position of a label in the text.

Example:

"globalOffsets": { "end": 1403, "start": 1398 }