Interpretability as a leakage detector: a case study in machine-learning-based exoplanet detection with TESS

This record corresponds to the accepted manuscript (post-print) of the following journal article published in Machine Learning: Science and Technology:

Interpretability as a leakage detector: a case study in machine-learning-based exoplanet detection with TESS

This article presents a low-resource and interpretable machine-learning framework for the classification of photometric light curves from the Transiting Exoplanet Survey Satellite (TESS). The proposed methodology combines engineered time-series features, Random Forest classification, and feature-importance analysis to identify potential planetary transit signals while examining the reliability and limitations of the resulting predictions.

The methodology is applied to TESS Sector 68. The dataset comprises confirmed exoplanet hosts as the positive class and targets without catalogued planets or candidates as the negative class. A Random Forest classifier is trained on statistical descriptors extracted from the normalized light curves, achieving a held-out ROC-AUC of 0.95, precision of 0.94, recall of 0.74, and an F1-score of 0.83 under the adopted sampling procedure.

A central contribution of the study is the identification of a potential confounding factor through feature-importance analysis. The two most influential variables, flux energy and its logarithmic transformation, are strongly related to the number of observational cadences and, consequently, to the observational baseline. A transit-independent ranking based on this feature alone achieves an AUC of 0.734, revealing that differences in the provenance and observational characteristics of the two classes may contribute substantially to the reported predictive performance.

To further examine the model's behavior, the pipeline is applied to 400 held-out candidates with observational baselines comparable to those of the negative class. The model identifies 61 candidates (approximately 15%) as probable planetary transits, with the highest-probability cases exhibiting recognizable periodic transit-like dips. These findings suggest that potentially meaningful transit information remains detectable, although the precise contribution of the identified confounding factor requires additional controlled evaluation.

The study emphasizes the importance of integrating interpretability into the validation of scientific machine-learning models. It provides practical recommendations involving provenance-matched datasets, baseline-invariant descriptors, controlled ablation studies, and systematic feature-importance audits to identify possible data leakage and misleading predictive relationships.

The main contribution is therefore not limited to exoplanet classification, but extends to the critical assessment of machine-learning reliability in scientific applications, demonstrating how interpretability techniques can be used to identify methodological limitations and support more robust and reproducible research.

The final published version is available at the publisher's website:

https://doi.org/10.1088/2632-2153/aeb3ca

This deposit is intended for scientific dissemination and preservation, subject to the publisher's applicable self-archiving policy.