Are bullies more productive? Empirical study of affectiveness vs. issue fixing time

1. Does the paper propose a new opinion mining approach?

Yes

2. Which opinion mining techniques are used (list all of them, clearly stating their name/reference)?

An approach to compute politeness of Wikipedia and Stack Overflow posts. C. Danescu-Niculescu-Mizil, M. Sudhof, D. Jurafsky, J. Leskovec, and C. Potts, “A computational approach to politeness with application to social factors,” in Proceedings of ACL, 2013. SentiStrength for sentiment analysis The authors built a machine learning classifier able to identify the presence of four basic emotions: JOY, LOVE, ANGER and SADNESS. This tool exploits: the Wordnet Affect label to obtain an affective label for each sentence’s words (see C. Strapparava, A. Valitutti et al., “Wordnet affect: an affective extension of wordnet.” in LREC, vol. 4, 2004, pp. 1083–1086.); Also, it uses the tool by De Smedt (see T. De Smedt and W. Daelemans, “Pattern for python,” The Journal of Machine Learning Research, vol. 13, no. 1, pp. 2063–2067, 2012.) to identify the mood of the sentence and its modality (i.e., the level of uncertainty). Finally, it also uses as input the output of the tool for the politeness of the sentences.

3. Which opinion mining approaches in the paper are publicly available? Write down their name and links. If no approach is publicly available, leave it blank or None.

SentiStrength: http://sentistrength.wlv.ac.uk Politeness tool: https://www.cs.cornell.edu/~cristian/Politeness.html Wordnet Affect label: http://wndomains.fbk.eu/wnaffect.html Pattern: http://www.jmlr.org/papers/v13/desmedt12a.html

4. What is the main goal of the whole study?

The authors empirically analyze >560k comments of the Apache projects’ Jira issue tracking system to understand the relation between human affectiveness and developer productivity. In particular, they extract affectiveness metrics for emotion, sentiment and politeness, then build regression models to understand whether these metrics can explain the time to fix an issue.

5. What the researchers want to achieve by applying the technique(s) (e.g., calculate the sentiment polarity of app reviews)?

Identify affectiveness in Jira comments

6. Which dataset(s) the technique is applied on?

>560k comments of the Apache projects’ Jira issue tracking system

7. Is/Are the dataset(s) publicly available online? If yes, please indicate their name and links.

No

8. Is the application context (dataset or application domain) different from that for which the technique was originally designed?

Yes (in part), for example in the case of SentiStrength.

9. Is the performance (precision, recall, run-time, etc.) of the technique verified? If yes, how did they verify it and what are the results?

Yes, the performance of the tool to identify the presence of four basic emotions in sentences was evaluated through Bootstrap validation (i.e., splitting a dataset in training and test set according to a given ratio (e.g., 90% training - 10% testing in this case) and generate N sets (1000 in this case) uniformly sampled with replacement from the initial dataset. Accuracy, precision, recall, and FM were then computed. Based on the sentiment, these metrics range between 0.73 and 0.89.

10. Does the paper replicate the results of previous work? If yes, leave a summary of the findings (confirm/partially confirms/contradicts).

No

11. What success metrics are used?

-

12. Write down any other comments/notes here.

-