KatanaG: Fragmenting Data Strategies to Enhance Knowledge Graph Creation from Large Datasets

Abstract. In recent years, the amount of data being generated has in-creased exponentially. Thus, a unified schema is needed to bring multiple data sources under a single format. For that reason, the use of knowledge graphs (KGs) has become commonplace. However, the problem of generating KGs efficiently has become a new problem. When creating a KG, multiple factors affect the creation process, like the size and heterogeneity of the input data and the complexity of the input mapping. When managing large data sources, a new problem arises:
how much memory will be needed? We propose KatanaG, a framework that encloses partition techniques to scale up the process of KG creation to complex scenarios, i.e., large data sources and complex mapping  assertions. As a result, memory usage and execution time are optimized. It is used along-side different knowledge graph creation engines to demonstrate that data source partitioning improves the knowledge graph creation process. The results indicate savings in execution time of one order of magnitude.