Tradeoffs in Discovering Latent Design Topics

Latent topic discovery has a rich history in software research. Statistical tools such as Latent Dirichlet Allocation (LDA) promise to uncover the latent topics in unstructured text. One such source is Stack Overflow, a rich if well-tapped vein of questions and answers about software development and design. However, while efficient and scalable, the topics produced can be hard for humans to interpret, and statistical tools are stochastic and unstable. We consider tradeoffs between degree of human supervision, algorith- mic efficiency, and their impact on the validity of latent topics extracted from unstructured texts about software design on Stack Overflow. We compare these algorithmic approaches to a more costly inductive coding approach. Software design is a good case study because design discussions typically require significant expert input in order to contextualize the abstract design information. We find that the semi-supervised (BERTopic) approach best balances cost, efficiency, and has the highest validity compared with the other approaches including LDA.