Optimizing Object Detection for Maritime Search and Rescue: Progressive Fine-Tuning of YOLOv9 with Real and Synthetic Data

The use of unmanned aerial vehicles for search and rescue (SAR) brings a series of advantages and reduces the time required to find survivors. It is possible to use computer vision algorithms to automate person detection, enabling a faster response from the rescue team. A major challenge in training image detection systems is the
availability of data. In the SAR context, it can be more challenging as datasets are scarce. A possible solution is to use a virtual environment to generate synthetic data, which can provide an almost unlimited amount of data already labeled. In this work, the use of real and synthetic data for training the model YOLOv9t in maritime search and rescue operations is explored. Different proportions of real data were used for training a model from the scratch and for transfer learning by fine-tuning the model after being pretrained with synthetic data generated in Unreal Engine 4, to evaluate the performance aiming to reduce the reliance on real-world datasets. The total amounts of real and synthetic data were kept the same to ensure fair comparison. Finetuning a model pretrained on synthetic data with just 10% real data improved performance by 13.7% compared to using real data alone. An important finding is that the best performance was achieved with 70‘% real data instead a model trained solely on 100‘% real data. These results show that combining synthetic and real data enhances detection accuracy while reducing the need for large real-world datasets.