The Influence of Code Comments on the Perceived Helpfulness of Stack Overflow Posts (Supplementary Material)

Question-and-answer platforms such as Stack Overflow have become an important way for software developers to share and retrieve knowledge. However, reusing poorly understood code can lead to serious problems, such as bugs or security vulnerabilities. To better understand the role of code comments in the perceived helpfulness of answers on Stack Overflow, we conducted an online experiment simulating a Stack Overflow environment (n=91). The results indicate that block comments in particular were perceived as significantly more helpful than uncommented source code, especially by novices. Novices also found code snippets with block comments more helpful than those with inline comments. Interestingly, other surface features, such as the position of an answer and its usefulness score, were considered less important. The content of Stack Overflow has been a major source for training large language models. Although AI-based coding assistants such as GitHub Copilot, which are based on these models, might change the way Stack Overflow is used, our findings have implications beyond this specific platform. First, the findings of this study may help improve the ongoing relevance of community-driven platforms like Stack Overflow that provide human advice and explanations of code solutions to complement AI-based support for programmers. Second, since chat-based tools can be prompted to generate code in different ways, knowing which properties influence the perceived helpfulness of code snippets is beneficial.