Appendix B. Robustness Checks
To ensure the reliability of our findings, we conducted several robustness checks:
- Hyperparameter sensitivity: Varying model hyperparameters by ±10% resulted in minimal changes to effectiveness scores (≤ 0.5 points), indicating stability in our results.
- Cross-validation stability: Increasing cross-validation folds from 5 to 10 maintained the ranking of top learning methods within each career stage and skill domain combination.
- Sample size stability: Reducing the sample to 80% of its original size through random sampling preserved the key patterns identified in the full dataset.
- Alternative models: Running parallel analyses with Random Forest, XGBoost, and LightGBM models yielded consistent patterns of learning effectiveness across career stages.
These checks confirm that our findings are not artifacts of specific methodological choices but represent robust patterns in the data.