﻿# Dataset for Comparative Evaluation of LLM Responses Across Role and Perspective Conditions

## Authors

Kunkel, Hartwig, Voss, Schütt & Gellrich

## Description

This repository contains the dataset accompanying the study:

**"Comparative Evaluation of LLM Responses Across Role and Perspective Conditions"**

The dataset contains multilingual prompt-response evaluation data for multiple large language models (LLMs) under different role and perspective prompting conditions.

Further methodological details and analysis will be available in the corresponding paper to be published on arXiv soon.

---

# Repository Structure

```text
.
├── llm_role_prompting_dataset.xlsx
├── README.md
├── Explanation_Indices.txt
├── 00_master_prompts.yaml
│
├── csv_exports/
│   ├── 00_master_prompts.csv
│   ├── 01_answers.csv
│   ├── 02_indices.csv
│   ├── 03_role_assistant_2nd_p.csv
│   ├── 04_role_cfo_2nd_p.csv
│   ├── 05_role_ngo_2nd_p.csv
│   └── 06_role_cfo_1st_p.csv
```

---

# File Descriptions

## `llm_role_prompting_dataset.xlsx`

Main Excel workbook containing all dataset tables and sheets in human-readable format.

---

## `csv_exports/`

Contains all Excel sheets exported individually as UTF-8 encoded CSV files for machine-readable processing and FAIR interoperability.

The numerical prefixes preserve the original sheet order from the Excel workbook.

---

## `00_master_prompts.yaml`

YAML representation of the master prompt set.

This file provides a structured and machine-readable representation of the prompts and is intended to facilitate automated processing, reproducibility, and interoperability across software environments.

---

## `Explanation_Indices.txt`

Documentation explaining the construction and interpretation of the three indices used in the dataset.

This includes methodological details relevant for reproducibility and downstream analysis.

---

# Dataset Contents

The dataset includes:

* Prompt texts in German
* English translations of prompts
* Responses and/or evaluations across multiple LLMs
* Different role prompting conditions
* Different perspective conditions
* Derived indices for comparative evaluation

---

# FAIR and Reusability Considerations

This repository is structured to support FAIR data principles:

* **Findable** through descriptive metadata and stable filenames
* **Accessible** via open formats (CSV, YAML, XLSX)
* **Interoperable** through UTF-8 encoding and machine-readable exports
* **Reusable** through documentation and explicit licensing

CSV files are encoded in UTF-8 to ensure compatibility with multilingual text processing.

---

# Keywords

* large language models
* prompting
* role prompting
* AI evaluation
* multilingual evaluation
* German language
* LLM benchmarking

---

# License

## CC-BY-4.0

This dataset, *"Dataset for Comparative Evaluation of LLM Responses Across Role and Perspective Conditions,"* is licensed under the Creative Commons Attribution 4.0 International License by Kunkel, Hartwig, Voss, Schütt & Gellrich.

https://creativecommons.org/licenses/by/4.0/deed

---

# Citation

Please cite the corresponding publication once available.

A DOI and citation metadata will be added upon Zenodo publication.
