Exploring Disagreements in XAI Techniques for Software Defect Prediction: An In-Depth Evaluation

Software defects can cause significant problems in software engineering, leading to reduced software quality, higher development costs, and increased maintenance requirements. Consistency is challenging when explaining the results of defect prediction models and the differences between explainers' views. Disagreement is frequent when the expected properties of the model are numerous. It is essential to understand better when and if the explanations produced by various post hoc explanation methods conflict. It is also important to understand how such conflicts are resolved in practice. This research describes and examines the disagreement problem in explainable machine learning. Machine learning models have been proposed to address this issue but to achieve accurate and reliable predictions. Feature engineering techniques, more specifically feature selection strategies are often applied to prepare the input data for the models. However, this can introduce issues related to explainability, as the feature selection process may need to be more transparent and well-documented. As a result, there has been increasing interest in explainable machine learning techniques to address these issues in recent years. This research paper investigates the impact of feature selection techniques on software defect prediction using explainable machine learning methods. The study aims to determine the effectiveness of various feature selection approaches, such as the Chi-square Test and Correlation Matrix with Heatmap, in increasing the accuracy and explainability of defect prediction models. The experiments are conducted on 33 datasets, and the results are analyzed to evaluate the performance of the models before and after applying feature selection techniques. The results shed light on the significance of feature selection in enhancing the effectiveness of software defect prediction in the context of explainable machine learning. Also, a study with 15 participants, mostly software developers, explores how feature selection impacts explainability (e.g., focusing on a small set of features) when explaining prediction outcomes. The results of our study indicate that the implementation of a specific set of features has the potential to enhance both the accuracy and explainability of the model. Our analysis has also revealed a notable research need in this area.