Mining API Usage Scenarios from Stack Overflow

1. Does the paper propose a new opinion mining approach?

Yes

2. Which opinion mining techniques are used (list all of them, clearly stating their name/reference)?

OpinerDSO, Senti4SD , SentiCR , and SentistrengthSE

3. Which opinion mining approaches in the paper are publicly available? Write down their name and links. If no approach is publicly available, leave it blank or None.

Senti4SD: https://github.com/collab-uniba/Senti4SD , SentiCR https://github.com/senticr/SentiCR, SentiStrengthSE https://www.sciencedirect.com/science/article/pii/S0164121218301675 are available. OpinerDSO is not.

4. What is the main goal of the whole study?

Generate task driving documentation by processing and summarizing StackOverflow posts.

5. What the researchers want to achieve by applying the technique(s) (e.g., calculate the sentiment polarity of app reviews)?

Calculate whether sentence are positive or negative about a specific API usage scenario.

6. Which dataset(s) the technique is applied on?

A manually made dataset of 4522 sentences labeled by 8 raters.

7. Is/Are the dataset(s) publicly available online? If yes, please indicate their name and links.

No.

8. Is the application context (dataset or application domain) different from that for which the technique was originally designed?

No, technique was designed with this application in mind.

9. Is the performance (precision, recall, run-time, etc.) of the technique verified? If yes, how did they verify it and what are the results?

Yes, the macro F1 score is compared. Showing that all tools score similar, with F1 scores ranging from .43 to .51. OpinerDSO: .495 Senti4SD: .510 SentiCR: .430 SentiStrengthSE: .454

10. Does the paper replicate the results of previous work? If yes, leave a summary of the findings (confirm/partially confirms/contradicts).

No.

11. What success metrics are used?

Macro F1 score.

12. Write down any other comments/notes here.

-