RADII: Bridging the Divide between Data and Infrastructure Management to Support Data-Driven Collaborations

We have witnessed a dramatic increase in national cyberinfrastructure resources to support data-driven research. Orchestrating these resources to enable the creation of col- laborative infrastructure capable of supporting data intensive activities is challenging. In this work we present RADII, a novel architecture and system that enables the provisioning and configuration of collaborative infrastructure by orchestrating data and infrastructure management in an integrated manner. We also introduce a cross-layer data annotation mechanism that together with Software-Defined-Networking (SDN) sup- port allows the embedding of user-defined performance policies into file metadata which are translated into executable optimal network plans. We have deployed RADII on a worldwide production testbed and demonstrated through experimentation that RADII can improve network throughput of data transfers by 28.2% as compared to conventional approaches.