SatiIndicator: Leveraging User Reviews to Evaluate User Satisfaction of SourceForge Projects

1. Does the paper propose a new opinion mining approach?

Yes

2. Which opinion mining techniques are used (list all of them, clearly stating their name/reference)?

SatiIndicator (LDA for topic modeling, improved recursive neural tensor network (IRNTN) for sentiment analysis)

3. Which opinion mining approaches in the paper are publicly available? Write down their name and links. If no approach is publicly available, leave it blank or None.

None

4. What is the main goal of the whole study?

to evaluate user satisfaction of a software project by analyzing user reviews with user opinions and emotions

5. What the researchers want to achieve by applying the technique(s) (e.g., calculate the sentiment polarity of app reviews)?

topic model: to cluster all aspects in a software genre into different topics and compute the weight of each topic, sentiment analysis: to calculate the sentiment strength of every aspect and pure attitude reviews,

6. Which dataset(s) the technique is applied on?

reviews of ten software genres before Nov. 21, 2015 Five master students read all of the 300 software projects reviews and give a satisfaction rank of all the 30 software projects in each software genre respectively

7. Is/Are the dataset(s) publicly available online? If yes, please indicate their name and links.

No

8. Is the application context (dataset or application domain) different from that for which the technique was originally designed?

sentiment analysis: yes, topic modeling: unsupervised

9. Is the performance (precision, recall, run-time, etc.) of the technique verified? If yes, how did they verify it and what are the results?

Yes, for sentiment analysis, compared with sentistrenth, for satisifaction ranking, compared with manual rank

10. Does the paper replicate the results of previous work? If yes, leave a summary of the findings (confirm/partially confirms/contradicts).

No

11. What success metrics are used?

precision, recall, f1 Precision@K (average precision at the top 50% software projects)

12. Write down any other comments/notes here.

-