Published January 1, 2022 | Version v1

Self-Healing Clinical Data Pipelines: An Agentic AI Framework for Autonomous Airflow Orchestration and Quality Anomaly Resolution

Description

The growing adoption of Electronic Health Records (EHRs), Internet of Medical Things (IoMT) devices, and cloud-based platforms in the healthcare industry makes clinical data pipelines increasingly complex and requires them to be monitored continuously and reliably. Today\\\'s pipeline management systems, however, are based on rigid rules, rigid workflow scheduling, and manual troubleshooting, and do not rapidly detect the failure and recover from data quality issues. This paper presents a new solution based on Agentic AI called Self-Healing Clinical Data Pipeline Framework (SHCDP-AI) for clinical data pipelines monitoring, management and self-recovery. Combines Apache Airflow orchestration, reinforcement learning, and intelligent software agents to identify anomalies, investigate their origins, and automatically take corrective action. From the results of experiments carried out using simulated data pipeline for healthcare, it is clear that SHCDP-AI system has higher accuracy of anomaly detection (97.2%), lower average recovery time (24 s), high availability of the pipeline (99.1%), low false alarm rate (2.3%), and high efficiency of the pipeline (98.4%) than the traditional approaches to monitoring. The results indicate the potential effectiveness of the proposed framework under autonomous clinical data management for the reliability, resilience and continuity of clinical data systems in the future.

Files

IJSET_V10_issue2_230.pdf

Files (279.2 kB)

Name Size
md5:df724ab50395cce01b2938664344736c
279.2 kB Preview Download

Additional details