Published November 2023
| Version v1
Dataset
Open
Zero-shot Bilingual App Reviews Mining with Large Language Models
Authors/Creators
- 1. EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Ales, France
- 2. EuroMov Digital Health in Motion, Univ Montpellier, IMT Mines Ales, Montpellier, France
Description
Classification
6000 English and 6000 French user reviews from three applications on Google Play (Garmin Connect, Huawei Health, Samsung Health) are labelled manually. We employed three labels: problem report, feature request, and irrelevant.
- Problem reports show the issues the users have experienced while using the app.
- Feature requests reflect the demande of users on new function, new content, new interface, etc.
- Irrelevant are the user reviews that do not belongs to the two aforementioned categories.
As we can observe from the following table, that shows examples of labelled user reviews, each review belongs to one or more categories.
| App | Language | Total | Feature request | Problem report | Irrelevant |
|---|---|---|---|---|---|
| Garmin Connect | en | 2000 | 223 | 579 | 1231 |
| Garmin Connect | fr | 2000 | 217 | 772 | 1051 |
| Huawei Health | en | 2000 | 415 | 876 | 764 |
| Huawei Health | fr | 2000 | 387 | 842 | 817 |
| Samsung Health | en | 2000 | 528 | 500 | 990 |
| Samsung Health | fr | 2000 | 496 | 492 | 1047 |
Clustering
1200 bilingual labeled user reviews for clustering evaluation. From each of the three applications and for each of the two languages present in the classification dataset, we randomly selected 100 problem reports and 100 feature requests. Subsequently, we conducted manual clustering on each collection of 200 bilingual reviews, all of which pertained to the same category.
| Garmin Connect | Huawei Health | Samsung Health | |
| #clusters in feature request | 89 | 74 | 69 |
| #clusters(𝑠𝑖𝑧𝑒≥5) in feature request | 7 | 9 | 11 |
| #clusters in problem report | 45 | 44 | 41 |
| #clusters(𝑠𝑖𝑧𝑒≥5) in problem report | 10 | 13 | 12 |
Files
dataset.zip
Files
(587.2 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:32c1fb0b093e7ce9e3c1b024dae9bfd6
|
587.2 kB | Preview Download |
Additional details
Identifiers
- arXiv
- arXiv:2311.03058
Related works
- Cites
- Preprint: 10.48550/arXiv.2206.14669 (DOI)
- Is new version of
- Dataset: https://zenodo.org/records/7261877 (URL)
- Is published in
- Conference proceeding: 10.1109/ICTAI59109.2023.00135 (DOI)
Software
- Repository URL
- https://github.com/Jl-wei/mini-bar
- Programming language
- Python