A visual method for comparing article-level clustering approaches

Understanding differences in the results obtained from different article-level clustering approaches is essential to select a suitable method for conducting a research project. The comparison is a source of understanding differences of approaches. However, previous comparative studies either only describe the differences between results, or only explain general reasons for differences. There is a lack of in-depth understanding of how different  approaches affect results. We propose a visual method for comparing clustering approaches that aims to explore the in-depth reasons that approaches affect results.

We constructed an overlay map for comparing the results obtained from two widely used article-level clustering approaches in the field of scientometrics, namely LDA and AC. Despite their popularity, these methods haven’t been compared yet. The overlay map intuitively shows the relations between clusters and topics, both one-to-one and one-to-many relations. It also shows clusters and topics that do not have any relations. The relations reflect the differences in the results obtained from the two clustering approaches. For instance, the one-to-many relations between clusters and topics reflect the different way of presenting knowledge organization (citation links and lexical similarities) of the same content by LDA and AC. It also reflects how the citation-based method (AC) and text-based clustering method (LDA) construct data and produce various results.

Our work provides a visual method for comparing article-level clustering approaches. It can help to understand differences in the results obtained from different approaches, thus facilitating their selection for practical applications. However, there are lots of possibilities for improving and extending the research. In this paper, we only compared two approaches. In the future, more approaches can be compared. In addition, based on understanding how different approaches’ properties affect results, learning the merits and shortcomings of approaches, then linking approaches and decisions to the purpose of the analysis would be an important step forward.