WEBIS-EXHIBITION-QUESTIONS-21

DOI: https://doi.org/10.5281/zenodo.5601979

We provide an online facility to replay the interactions of the participants. Please have a look at the dataset's page (via the DOI) for an up-to-date URL.

study.jsonl
===========
The main file of this dataset is the study.jsonl, which contains the data collected for each of the 63 participant as one JSON object each, separated by line breaks. THe JSON in each line has the following structure:

```
{
  "id": "<ID for this participant>",
  "language": "<language of the audio clips, either 'DE' for German or 'EN' for English>",
  "perspective": "<perspective of the audio clips, either 'narrator' or 'personal'>",
  "duration": <time it took the participant to finish the study in milliseconds>,
  "tasks": [
    {
      "name": "<theme of the task, one of 'room', 'rug', 'desk', 'bauhaus-lamp', 'wall-hanging', or 'antechamber'>",
      "length": "<length of the audio clip, one of 'short', 'medium', or 'large'>",
      "text": "<text of the audio clip>",
      "questions": [
        {
          "text": "<text the participant typed in>",
          "annotations": [<list of annotations, see section 'Annotations' below>],
          "events": [ <list events that happened while working on this question, see section 'Events' below> ],
        }, ...
      ]
    }, ...
  ],
  "log": [ <list of all events, see section 'Events' below> ]
}
```


Annotations
-----------
Each of the 849 questions has been manually annotated with the following attributes. The annotations object of a question object contain the respective tags of the set annotations (numbers give the frequency of the respective tag).
- aboutness-concept (91): The question asks for information about a concept (e.g., the Bauhaus, Modernism)
- aboutness-design-choices (131): The question asks for reasons for specific design choices, how or why a school of design would choose, or what typical design choices are
- aboutness-developments (66): The question asks for information on how something changed or where people or objects went
- aboutness-historical (68): The question asks for information about the historical, political, or social context at the time of the room's creation
- aboutness-object (458): The question asks for information about an object that has been or is in the room
- aboutness-person (94): The question asks for information about a person
- aboutness-reconstruction (35): The question asks for information about the reconstruction (especially authenticity)
- aboutness-room (190): The question asks for information about the room or its conceptualization
- context-heard (549): The question is likely based on something the participant heard
- context-marked-object (76): The question likely refers to the marked object (this, that, ...)
- context-task-object (389): The question refers to the object of the task (not in task 1, the anteroom in task 6)
- context-visible (336): The question is likely based on something the participant saw
- expects-description (121): The question asks for a description or definition
- expects-fact-color-name (2): The question asks for the name of a color
- expects-fact-concept-name (8): The question asks for the name of a design concept, style, or kind of object
- expects-fact-electrical (4): The question asks for the wattage or voltage of an object
- expects-fact-location (5): The question asks for the name of a location (e.g., where someone is from)
- expects-fact-material-name (50): The question asks for the name of the material of some object
- expects-fact-number-exemplars (3): The question asks for the number created or sold of some object
- expects-fact-number-people (5): The question asks for the number of people involved in some action
- expects-fact-other-name (2): The question asks for the name of something not covered by other "expects-fact" annotations
- expects-fact-other-number (4): The question asks for a number not covered by other "expects-fact" annotations
- expects-fact-person-name (31): The question asks for the name of a person (designer, ...)
- expects-fact-price (42): The question asks for a monetary value
- expects-fact-size (10): The question asks for the size of some object or some
- expects-fact-year (51): The question asks for a year or time span
- expects-interaction (9): The question asks for an interface action (repeat, interacting with the room, showing something related, ...)
- expects-reasoning (265): The question asks why something happened, why something is as it is, or what the function of something is
- personal-address (13): The question is posed directly to the object


Events
------
The 'log' attribute of each assignment contains the list of event records for all events during the study participation, and the 'events' attribute for a question contains the list of event records for all events while working on that question (see the paper). Each record has the following generic attributes and may have additional ones depending on the value of `event`.

Generic event attributes:
- time: Number of milliseconds that passed since the participant started the study.
- event: The event name, see below.
- task: The active task (0 to 5).
- question: The active question (starting with 0); not set for the task-next and task-previous events.

Event explanations and additional attributes:
- audio-ended:
  The audio finished playing.
- audio-pause:
  The participant paused the audio (by pressing the button, changing a question, or changing task).
  - position: Current position of the audio clip in min:sec.
  - by-button: The event was triggered by pressing the button.
  - by-question: The event was triggered by changing a question.
- audio-play:
  The participant started the audio (by pressing the button).
  - position: Current position of the audio clip in min:sec.
- audio-reset:
  The participant reset the audio to 0:00 (by pressing the button).
- camera-move:
  The participant moved the camera of the 3D panorama.
  - x: longitude.
  - y: latitude.
- comment-focus:
  The participant focused the comment text box of task.
- comment-set:
  The participant changed the text for the comment of the task.
  - value: The changed text
- comment-unfocus:
  The participant unfocused the comment text box of the task.
- done:
  The participant finished the study by pressing the button.
- marker-set:
  The participant positioned the marker for the task.
  - x: X-coordinate of the marker in 3D panorama space.
  - y: Y-coordinate of the marker in 3D panorama space.
  - z: Z-coordinate of the marker in 3D panorama space.
- question-add:
  The participant added a new text input field for another question.
- question-active:
  The following events are considered to happen while working on the question and task.
  - duration: Time in milliseconds until the last event while the question stays active
- question-focus:
  The participant focused the text box of the question and task.
- question-set:
  The participant changed the text for the question and task.
  - value: The changed text
- question-unfocus:
  The participant unfocused the text box of the question and task.
- task-next:
  The participant switched to the next task. The task number specifies the current task (before the change).
- task-previous:
  The participant switched to the previous task. The task number specifies the current task (before the change).


Audio clips
===========
This part of the dataset is provided compressed as audio-clips.zip.

The audio clips (and their SSML fragment source code) are located in:

  tasks/generated-<perspective>-<language>/<task-name>-<length>.{mp3,txt}

The file `tasks/tasks.txt` lists how each such file is composed from the respective snippets in `snippets/<perspective>/<language>/<snippet-name>.txt` by specifying space-separated blocks (corresponding to paragraphs) of comma-separated snippets (identified by their snippet-name). The file `tasks/voices.txt` specifies which Amazon Polly voice to use for which task.