Emotions Extracted from Text vs. True Emotions–An Empirical Evaluation in SE Context
1. Does the paper propose a new opinion mining approach?
Yes
2. Which opinion mining techniques are used (list all of them, clearly stating their name/reference)?
NTUA-SLP, a deep learning model for emotion detection and supporting transfer learning.
3. Which opinion mining approaches in the paper are publicly available? Write down their name and links. If no approach is publicly available, leave it blank or None.
NTUA-SLP https://github.com/cbaziotis/ntua-slp-semeval2018
4. What is the main goal of the whole study?
Compare self-reported emotions to emotions detected from the text in Slack from developers during a long running SE-ing project.
5. What the researchers want to achieve by applying the technique(s) (e.g., calculate the sentiment polarity of app reviews)?
Calculate the emotional scores of text in public channels in Slack.
6. Which dataset(s) the technique is applied on?
135k sentences extracted from Slack, of which 3,100 have been manually labeled by several coders to label which emotion is associated to each sentence.
7. Is/Are the dataset(s) publicly available online? If yes, please indicate their name and links.
No.
8. Is the application context (dataset or application domain) different from that for which the technique was originally designed?
Partly, yes the domain is different, however, the technique supports transfer learning.
9. Is the performance (precision, recall, run-time, etc.) of the technique verified? If yes, how did they verify it and what are the results?
Yes, authors use MAE (mean average error) and Pearson's to validate performance of the trained model.
10. Does the paper replicate the results of previous work? If yes, leave a summary of the findings (confirm/partially confirms/contradicts).
No.
11. What success metrics are used?
MAE and Pearson's.
12. Write down any other comments/notes here.
-