Published March 29, 2023
| Version 1.0
Dataset
Open
GPT Capabilites for Extracting Tasks From Textual Process Descriptions
Authors/Creators
- 1. Technical University of Munich
- 2. Austrian Center for Digital Production
Description
This dataset provides three tables which evaluate the capabilities of GPT 1, GPT 2, GPT 3, and GPT 4 regarding the extraction of tasks from https://doi.org/10.5281/zenodo.7783492 dataset.
The performance of the LLMs is measured by calculating a range of similarity metrics:
- Extracted number of tasks from text vs. extracted number of tasks from Model
- Semantic Text Similarity: Contextual and Non-Contextual between extracted sets of tasks
- Semantic Text Similarity: Contextual and Non-Contextual between extracted individual tasks
- Similarities and Prevalence for length restricted extracted labels
- Similarities and Prevalence for augmented texts (each text has been paraphrased by 9 different paraphrasing methods)
Files
Files
(412.1 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:f12a529b368b36395cc7af4b16268852
|
99.7 kB | Download |
|
md5:cd1033373d40ec41c015d3cbf4dea033
|
208.2 kB | Download |
|
md5:8c3b3a5b4d1583cd6a196940a713f945
|
104.2 kB | Download |