Trustworthy AI-based Multi-Modal Data Processing for Cultural Heritage Monitoring and Preservation

ARGUS addresses growing threats to remote built heritage sites — including climate change, natural disasters, seismic activity, and human activity — through an integrated AI-powered monitoring and preservation framework validated across five European pilot sites. Realizing its objectives depends fundamentally on the quality, consistency, and accessibility of the underlying data. However, CH data is inherently heterogeneous and multi-modal, posing significant challenges in format interoperability, metadata completeness, spatial and temporal coverage, and FAIR compliance. Consider Delos Island, where existing data comprised 53 disparate datasets across four formats with inconsistent coordinate systems, while real-time measurements were limited to sparse point-wise observations with significant temporal gaps, making it difficult to directly apply AI solutions for heritage monitoring. To address these challenges, we present two complementary methods. First, a CH data processing pipeline covering seven systematic stages from data collection through publication transforms heterogeneous multimodal data into a unified FAIR-compliant GeoPackage (GPKG) database with a consistent WGS84 coordinate reference system. Geospatial coverage is extended through nearest-neighbour spatial interpolation, while real-time sensor data are aligned through temporal imputation to ensure continuous time-series records. The pipeline is operationalized through a DataOps framework, enabling modular, automated, and continuously monitored data processing across all pilot sites. Second, a Retrieval- Augmented Generation (RAG)-based LLM application grounds language model outputs in verified information retrieved from external knowledge bases such as WikiData for automated attribute annotation and translates natural language queries into SQL statements executed against the internal GPKG database for intuitive data access and reliable reasoning. All outputs are labelled and source-referenced to ensure transparency and responsible AI usage. 

Initial results demonstrate successful processing of both existing and real-time data across five pilot sites, with the RAG-based LLM showing promising capability in generating traceable metadata and supporting natural language queries, collectively advancing ARGUS’s capacity for scalable, trustworthy AI-driven cultural heritage preservation.