Published March 29, 2023
| Version 1.0
Dataset
Open
GPT Capabilites for Extracting Tasks From Textual Process Descriptions
Creators
- 1. Technical University of Munich
- 2. Austrian Center for Digital Production
Description
This dataset provides three tables which evaluate the capabilities of GPT 1, GPT 2, GPT 3, and GPT 3.5 regarding the extraction of tasks from https://doi.org/10.5281/zenodo.7783492 dataset.
The performance of the LLMs is measured by calculating a range of similarity metrics:
- Extracted number of tasks from text vs. extracted number of tasks from Model
- Semantic Text Similarity: Contextual and Non-Contextual between extracted sets of tasks
- Semantic Text Similarity: Contextual and Non-Contextual between extracted individual tasks
- Similarities and Prevalence for length restricted extracted labels
- Similarities and Prevalence for augmented texts (each text has been paraphrased by 9 different paraphrasing methods)
Files
Files
(378.2 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:a7460424aacbc405c58789f98799ae7d
|
167.2 kB | Download |
|
md5:9e79823732a9e9b4ae78a865c2e7b6e7
|
116.6 kB | Download |
|
md5:224e0902012eb6ca6fa7a632883156fd
|
94.5 kB | Download |