Published June 2024 | Version Pre-print
Conference paper Open

Scalable Data Profiling for Quality Analytics Extraction

  • 1. ROR icon Institute of Communication and Computer Systems
  • 2. ROR icon Tecnalia
  • 3. ROR icon Centro Tecnológico de Investigación, Desarrollo e Innovación en tecnologías de la Información y las Comunicaciones (TIC)
  • 4. ROR icon Hellenic Telecommunications Organization (Greece)
  • 5. ROR icon Centre for Research and Technology Hellas

Description

In today’s modern society, data play an integral role in the development global industry, since they have become a valuable asset for companies, institutions, governments, and others. At the same time, data generated daily, at a global scale, require significant resources to pre-process, filter and store. When it comes to acquiring such stored
data, it is essential to understand which dataset fits to the needs of the user beforehand. One particularly important factor is the quality of a dataset, which could be determined based on a series of quality related attributes generated by it. Such attributes constitute “Profiling”, the process of obtaining information from a data sample, related to the complete dataset’s quality. However, in the era of Big Data, the ability to apply profiling techniques in complete large datasets should also be considered,
in order to obtain complete quality insights. This paper attempts to provide a solution for this consideration by presenting “DaQuE”, a scalable framework for efficient profiling and quality analytics extraction in complete datasets of all volumes.

Files

Scalable_Data_Profiling_for_Quality_Analytics_Extraction.pdf

Files (328.4 kB)

Additional details

Funding

European Commission
DATA Monetization, Interoperability, Trading & Exchange Grant agreement ID: 101092989