How Effective are Nature-Inspired Optimisation Techniques in Hyperparameter Tuning of Machine Learning Models

Hyperparameter optimization is crucial for enhancing the performance of machine learning models. This study explores the practicality of three nature-inspired optimization techniques - Bald Eagle Optimizer (BEO), Particle Swarm Optimization (PSO), and Mother Tree Optimization (MTO) for tuning the hyperparameters of Random Forest and Support Vector Machine (SVM) models. To ensure broad generalization, five datasets, including both image-based and tabular data, were utilized. The results reveal that while Optuna consistently balanced accuracy and training time effectively, the performance of other techniques varied across datasets. This research provides insights into the effectiveness of these optimizers and evaluates whether their use is practical and beneficial.