Published December 15, 2023 | Version v1
Conference paper Open

Ocean Data Quality Assessment through Outlier Detection-enhanced Active Learning

  • 1. ROR icon University of Amsterdam

Contributors

Data manager:

Supervisor:

Description

Ocean and climate research benefits from global ocean observation initiatives such as Argo, GLOSS, and EMSO. The Argo network, dedicated to ocean profiling, generates a vast volume of observatory data. However, data quality issues from sensor malfunctions and transmission errors necessitate stringent quality assessment. Existing methods, including machine learning, fall short due to limited labeled data and imbalanced datasets. To address these challenges, we propose an Outlier Detection-Enhanced Active Learning (ODEAL) framework for ocean data quality assessment, employing Active Learning (AL) to reduce human experts’ workload in the quality assessment workflow and leveraging outlier detection algorithms for effective model initialization. We also conduct extensive experiments on five large-scale realistic Argo datasets to gain insights into our proposed method, including the effectiveness of AL query strategies and the initial set construction approach. The results suggest that our framework enhances quality assessment efficiency by up to 465.5% with the uncertainty-based query strategy compared to random sampling and minimizes overall annotation costs by up to 76.9% using the initial set built with outlier detectors.

Files

2023.conference.bigdata.camera.pdf

Files (1.4 MB)

Name Size Download all
md5:aaefa8a35fdb7d7517bc14e33e09cbe8
1.4 MB Preview Download

Additional details

Identifiers

Funding

ENVRI-FAIR – ENVironmental Research Infrastructures building Fair services Accessible for society, Innovation and Research 824068
European Commission
Blue-Cloud 2026 – A federated European FAIR and Open Research Ecosystem for oceans, seas, coastal and inland waters 101094227
European Commission
CLARIFY – CLoud ARtificial Intelligence For pathologY 860627
European Commission
ARTICONF – smART socIal media eCOsytstem in a blockchaiN Federated environment 825134
European Commission