Int5Gent: An integrated end-to-end system platform for verticals and data plane solutions beyond 5G

— Int5Gent targets the integration of innovative data plane technology building blocks under a flexible 5G network resource, slice and application orchestration framework, providing a complete 5G system platform for the validation of advance 5G services and Internet of Things (IoT) solutions. The platform can act as the enabler for the transition beyond the current 5G networking capabilities allowing novel and state-of-the-art data transport and edge processing solutions to be evaluated under a cutting-edge network orchestration framework, with intelligent service allocation and management capabilities. A sample of the envisioned technologies include: flexible multi-Radio Access Technology (multi-RAT) baseband signal processing, millimeter Wave (mmWave)technology solutions at 60GHz and 150GHz bands, hardware-based edge processor with Time Sensitive Networking (TSN), Graphical Processing Unit (GPU)processing capabilities, and elastic Software Defined Networking (SDN)-based photonic data transport. The integration of the technology blocks is performed as part of an overall architecture that promotes edge processing and is orchestrated by a Network Function Virtualization Orchestrator (NFVO) compatible framework with edge node extensions at the network layer and an overlay vertical services application orchestrator at the user plane layer.


INTRODUCTION
The deployment of advanced 5G infrastructures that extend from the data plane technology blocks to the control plane and application deployment layer, is of paramount importance for the proper evaluation and showcasing of innovative infrastructure (i.e. data transport, switching and processing) solutions able to meet demanding functional requirements stemming from vertical applications, thus extending even beyond the current 5G capabilities. From the networking point of view, the increasing interest in new use cases with strict latency requirements, fast service deployment times, dynamicity and trustworthiness generates a clear trend towards distributed network models implemented through the edge computing concept [1]. Essentially, this concept alters the typical and simple-structured cloud-based connectivity model (access-core-cloud) to a mesh type model in which some functions must be executed at the edge part of the network and provide feedback to the attached end user devices, while portions of data can also be passed to the cloud. The complexity increases further by considering different types of edge nodes that may span from simple gateway servers to minidata centers (DCs), thus having different connectivity requirements. From the data plane point of view, the new technology building blocks should enable the 5G network infrastructure to provide the flexible high capacity and expandable connectivity between 5G terminals and edge computing nodes as well as among mobile edge, computing and content delivery nodes, and the core infrastructure supporting also the legacy cloud computing level. For this purpose, the move towards higher operating bands (V, W and even D-band) and the deployment of photonic interconnection solutions are necessary and require efficient elements for data distribution and demanding RF electronic system designs. In Fig. 1. The architecture of the Int5Gent system platform from data plane to the control and user plane and with appropriate interfaces and end user connections addition, the service level requirement for low latency in combination with new intelligent processing algorithms at the edge node denotes the deployment of edge processing units able to handle such services in real time and flexibly in terms of resource usage. Moreover, the move towards graphical processing unit (GPU) assisted edge processing gains increasing interest for its usage in various applications requiring extensive parallelism as for example in Artificial Intelligence (AI)-based applications and related video processing and data analytics services [2].
The overall rationale for Int5Gent is to deploy a holistic 5G system platform that combines new technological blocks for the data plane infrastructure orchestrated by flexible Physical Network Function (PNF) and Virtualized Network Function (VNF) instances over a generalized Network Functions Virtualization (NFV) Infrastructure (NFV-I) that is extended to edge computational, storage and networking resources. An overlay application orchestrator for the vertical services allows a pragmatic approach for the services' deployment and management. This article highlights the main concept and targeted innovations by the Int5Gent project focusing on specific solutions that enable the move beyond the 5G capabilities.

II. THE INT5GENT ARCHITECTURE
The overall architecture and the structure of the envisioned 5G system platform is depicted in Fig.1. At the data plane level, emphasis is given on integration of enabling technology blocks and components at the optical fronthaul and cell site (5G terminal) segments assuming different types of fronthaul data transport solutions, from typical digital schemes, to bandwidth efficient analogue radio over fibre formats and to advance novel schemes such as Σ-Δ modulation. For the edge node, advance hardware-based processing solutions are implemented with special emphasis on GPU accelerators for services requiring parallel processing. Central processing unit (CPU) hardware solutions are also considered for intensive serial data processing applications. The flexible edge node engine provides multiple interfaces depending on bandwidth and latency needs, supporting also intense computation tasks at the edge and at the Base-Band Unit (BBU) pool level.
At the control plane level, an extended ETSI NFV Management and Orchestration (MANO) architecture framework is developed featuring: • Joint and optimised provisioning of transport network and edge computing resources, • Multi-tenancy support through network slicing to improve the efficiency in the utilization of fronthaul resources with integration between NFV MANO functions for service orchestration and SDN-based configuration of the transport networks.
The interfacing with the data plane relies on appropriate (i.e. flexible, common and easily extendable) mapping of high-level orchestration operations into low-level, hardwaredependent operations using proprietary protocols operating on interfaces between the SDN controller and the network aggregation and switching elements, and also between the PNF manager and the adaptive transceiver components.
The user plane implements the orchestration and management of vertical applications and user defined service functions, acting as interface between end users, network resources and high-level management tool for the infrastructure owner(s). The key functionalities of the Vertical Application Orchestrator (VAO) are • the registration and lifecycle management of services/functions through adaptive application graphs in the form of chainable application components, • the deployment of services through slice instantiation requests sent over OpenStack interfaces to the MANO slice manager, • the definition of policies for governing the lifecycle of the service function entities being deployed, • the provision of data analytics software and the visualization of results through customizable dashboard.
Two interfaces are implemented, namely: a VAO to Graphical User Interface (GUI) for the interaction of the service providers, application component developers and service function end users with the VAO, and a Data Analytics Dashboard for providing monitoring and control information to the infrastructure owner and service providers.
The above structure provides a fully operational 5G system platform from the user end to the data plane able to host all stakeholders of a 5G ecosystem and will be validated by showcasing unique and dynamic synergies among the multiple Int5Gent technology providers, service providers, application developers and operators, aiming to form the first coordinated effort for a holistic interoperable multi-Radio Access Technology (multi-RAT) cross-split environment.

III. DATA PLANE TECHNOLOGY BUILDING BLOCKS
The key technological blocks included in the data plane of Int5Gent infrastructure are briefly discussed through this section. These blocks are grouped into three categories: • transport options for bandwidth-efficient optical Control/Data Unit (CU/DU) and Radio Unit (RU) interconnection, • millimeter wave arrays and subsystems for 5G terminal nodes, • WDM-enabled optical segments for flexible edge network.
A. Edge Node-Radio frontends optical interconnection Int5Gent supports a flexible optical transport layer, interconnecting its baseband processor platform with different radio terminal nod technologies. Depending on network and service-level requirements, digital, sigma-delta-over-fiber and analog optical links will serve as the bandwidth efficient transport layer between the edge node (described in section IV) and the sub-6GHz, V-band and D-band radio frontends. In more detail: 1. Standardized digital transport: Int5Gent will utilize the typical digitized format as defined by 3GPP standards, while also supporting open Radio Access Network (O-RAN) fronthaul standards. Int5Gent standardized digital infrastructure will support split option 7.2, allowing for high throughput and low latency fronthaul connectivity, based on the GPU acceleration solutions for baseband tasks [3]. Towards this implementation, high-speed transceivers (SFP, SFP+, QSFP) will enable the communication interface between the baseband processor and the radio terminal nodes.
2. Digital Signal Processing (DSP)-assisted analog Radioover-Fiber (ARoF): Int5Gent aims to deliver a DSP-assisted Analog Intermediate Frequency-over-Fiber (A-IFoF) solution [4], allowing for the implementation of centralized DSP functions which support the needs of Fixed Wireless Access (FWA) concepts, based on directional mmWave outdoor deployments. A-RoF links will be capable to support scalable modulation bandwidths from 100MHz to multiple GHz, for both single-carrier and multi-carrier radio waveforms and they will be implemented in the optical domain through Electroabsorption Modulated Laser (EML)-based and commercial offthe-shelf (COTS) linear photoreceivers. Centralized DSP running at the Int5Gent edge node will address the impact of optoelectronic and radio frontends interfaces and will allow for mutli-Gigabit IFoF/D-band transmission rates.
3. Sigma-Delta modulation over fibre SDMoF: Combining the advantages of ARoF and Digital Radio-over-Fiber (DRoF) schemes, Int5Gent infrastructure will also deploy a sigma delta-over-fiber (SDoF) transport layer. This Digital-to-Analog Converter (DAC)-free implementation will allow for the generation of a digital communication link, immune to nonlinearities, which will be interconnected with simple and power efficient Remote Radio Heads (RRHs) [5]. Multistream, bit-interleaved SDMoF interfaces will be developed for both CU/DU and RRU stations. They will be then deployed via Field Programable Gateway Array (FPGA)-based implementations and they will be combined with in-house developed sub-6GHz active antenna units.

B. mmWave and beyond radio blocks towards evolved RAN
Aside from the developments on optical transport technologies, innovative mmWave radio technological blocks will support Int5Gent's network architectural proposal. These include the use of V-band mesh nodes for Point-to-Multipoint (P2MP) edge networks as well the development of sub-THz nodes equipped with 120-170 GHz radio frontends for Fixed Wireless Access (FWA) scenarios.

D-band terminal nodes:
Targeting the frequency band from 120 to 170 GHz, Int5Gent will develop beyond 5G radio terminal nodes for Point-to-Point links in FWA deployments. These radio nodes will be comprised of D-band antenna modules, Local Oscillator (LO) analog processing blocks, data interfaces, mixers, and amplifier blocks. Following innovative system integration and packaging approaches [6], different versions of compact integrated D-band frontends will be then combined with the appropriate electro-optic interfaces for supporting converged analog fiber/wireless IFoF/D-band links.

V-band P2MP connectivity for edge networking:
Int5Gent will deliver a P2MP edge network approach based on a SDN-enabled 57-64 GHz mesh node interconnected with several client nodes. Exploiting the beam-steering gains of multiple-element antenna units, network arrangements with beam-steering sectors will allow for GHz-scale operation bandwidths for wireless distances above 100m.

C. Flexible optical edge layer with "on-demand" bandwidthcapacity steering functionalities
To fully exploit the innovations of its optical transport layer as well the advanced radio connectivity blocks, Int5Gent will design and develop a truly flexible 5G Centralised-Radio Access Network (C-RAN) architecture based on WDMenabled optics (tunable SFPs and laser sources) and photonicbased Reconfigurable Add/Drop Multiplexers (ROADMs). In this way, flexible optical bandwidth steering functionalities will be delivered in the optical edge layer of Int5Gent architecture, allowing thereby to dynamically allocate any optical fronthaul interface through spatial or wavelength multiplexing, by dropping/routing more than one optical channel to the sub-6GHz, mmWave and D-band terminal nodes.

IV. EDGE PLATFORM DESIGN
At the edge node, Int5Gent introduces (i) a flexible fronthaul RoF adapter to bridge a rich variety of radio interfaces with (ii) an advanced Edge Box processing platform, as shown in Fig. 2. By selectively activating/deactivating and connecting each of the interfaces to the RRU, the Edge Box will be capable of satisfying the vastly varying needs of Int5Gent use cases, supporting true flexibility between highcapacity, bandwidth, spectral-efficiency, and low end-to-end latency-budget depending on the selected functional split or fronthaul interface.
The fronthaul RoF adapter supports most existing fronthaul splits, i.e., digital (high and how layer), sigma-delta (Σ-Δ), and analog interfaces. The analog DSP engine will be developed on Xilinx's Radio Frequency (RF) System on Chip (SoC) platform [7], while the Σ-Δ DSP will be based upon an extension of IMEC's previously demonstrated all-digital implementations. The baseband processing stack intended for the digital RoF will be implemented using specialized hardware and software blocks provided by NVIDIA within the Edge Box, which is explained next.
The role of the Edge Box is two-fold. First, it provides a fully flexible Ethernet compliant baseband processor engine, shown as virtual BBU (vBBU) in Fig. 2. The vBBU utilizes high-speed optical SFP-based Ethernet interfaces to receive Ethernet frames from the 5G core (shown on the right to the Edge Box in Fig. 2); these Ethernet frames are then translated into raw bit-streams, which are transmitted through a set of reconfigurable fronthaul interfaces (one for each communication protocol stack, i.e., analog, digital, and Σ-Δ RoF, as shown on the left to the Edge Box in Fig. 2) towards the respective RRU. Secondly, the Edge Box also acts as a low latency edge processing platform for critical 5G services, such as AI inference and real-time video analytics.
To support both roles the Edge Box relies on state-of-theart edge network processors, also known as data processing units (DPUs), provided by NVIDIA (e.g., NVIDIA Bluefueld-2X ® ), as shown in Fig. 2. The NVIDIA DPUs combine highperformance (i.e., 100 GbE and 200 GbE) network interface cards with programmable Artificial Intelligence Enhanced Computing (ARM) CPU cores and GPUs on a single powerful SoC. NVIDIA's Aerial Software Development Kit (SDK) ® is exploited to implement GPU-accelerated baseband processing functions for the vBBU using a set of relevant Compute Unified Device Architecture (CUDA) modules (i.e., cuVNF and cuBB) deployed and managed by Kubernetes through the Int5GEnt NFVO. Apart from the BBU processing, Int5GEnt's vertical application orchestrator exploits the remaining processing power of the Edge Box (i.e., additional GPUs, ARM cores embedded in the Smart Network Interface Controller (SmartNIC), and regular CPU cores) to deploy and manage latency-sensitive edge applications, such as Ultrareliable low-latency communication (URLLC) 5G services, video encoding/processing, AI-based algorithms, etc.

V. SERVICE DEPLOYMENT AND NETWORK ORCHESTRATION
The service layer of the Int5Gent system is based on a Vertical Application Orchestrator (VAO) [8] cooperating with an NFV Orchestrator (NFVO) [9] and a tool for network slice management. The combination offers a powerful, complete and fully operational framework for the deployment of end-user service function requests over the attached data plane infrastructure. The overall architecture is depicted in Fig.3.
The VAO [8] manages the lifecycle of the applications and service function requests. These are initiated through a GUI that essentially constructs the application graph of the requested service and the properties of each software component. The graph is then passed to the deployment manager which creates the connection requests to the NFV orchestrator in the form of slice intents. Once the network connections for the requested service are established and the required resources are reserved, the generated slice instance is fed back in order to launch the service. The policy engine monitors the lifecycle of the service according to the pre-set properties and the dynamic policy criteria that may apply. In addition, an intelligent orchestration mechanism is developed and interfaces with the network slice and service monitoring data bases of the network orchestrator in order to provide policy related actions and also collect network state information which is in turn passed to the data analytics engine. The analysed monitoring data are visualised via a operation dashboard and can be used further in order to trigger actions.
The lifecycle of the requested network slices by the VAO is handled through a Network Slice Manager, under the coordination and the guidelines received by the VAO. Internally, the Network Slice Manager provides the logic to handle the composition and decomposition of network slices, optimizing the sharing and the dynamic scaling of their components based on security or service priority constraints identified by the VAO. Exploiting the interaction with the NFVO, network slices will be dynamically provisioned, customized, monitored and operated according to the runtime requirements of the service applications.
At the control plane, the NFVO [9] will manage the allocation of the resources across the available edge, fog and cloud nodes, potentially distributed in multiple administrative domains and controlled through technology-specific Virtualized Infrastructure Managers (VIMs). In particular, VIMs will be specialized according to the virtualization capabilities offered in each domain (e.g. implementing Kubernetes for the orchestration of containers, distributed serverless platforms for the execution of event-based functions, or fog-based virtualization solutions like Eclipse Fog05). The instantiation of the service functions will be complemented through the provisioning of custom network paths to guarantee the required interconnectivity between edge, fog and cloud nodes, automatically tailored to the dynamicity of the service deployment.
The provisioning of network connectivity is handled through SDN controllers acting as Wide area Infrastructure Manager (WIM) and specialized for the network technologies available in the NFV Infrastructure, e.g. at the fronthaul/backhaul segments [10]. The interaction between NFVO and SDN controller will follow the principles of the latest interfaces defined in the ETSI NFV MANO framework. In particular, it will allow to gather information about the network topology as input for the NFVO resource allocation algorithms, to collect monitoring data about network performance as input for cognitive networking strategies, and to establish network paths for the inter-/intra-site traffic flows, in compliance with the virtual networking approach adopted at the different VIMs and at the edge of the related PoPs.

VI. CONCLUSIONS
This concept paper describes the vision, architecture and implementation aspects of the EU Horizon-5GPPP Int5Gent project. The main goal of Int5Gent is the deployment of an end-to-end system platform that extends from the data plane to the service layer providing a unique test-bed for the evaluation Fig 3 The Int5Gent network control architecture of applications over innovative physical layer technology solutions. The envisioned platform considers the common 5G digital connectivity interfaces as well as the next generation mmWave and the bandwidth efficient ARoF and Sigma-Delta format-based schemes. A unique edge node interfaces seamlessly with any scheme offering the required processing power to each fronthaul processing interface. The node combines efficiently the BBU functionalities with CPU and GPU based processing for hosting latency sensitive applications at the edge. Finally, it offers the required connectivity to the core. An NFV/SDN network orchestration layer that extends to the edge node and access, manages the resources and network connections while an intelligent overlay application orchestration module interfaces with the application developers and vertical end-users to manage and deploy the services over the network. The offered end-to-end capabilities of the Int5Gent platform extend beyond those of 5G enabling for first time innovative data plane solutions to interface efficiently with the core network and also be tested with real overlay managed services.
The overall platform is intended to be deployed over two extended testbeds which include actual field deployed segments while validation and showcasing is planned with 3 advanced use cases covering a number of service scenarios related to multiple vertical sectors as well as innovative applications for smart IoT networked devices, highlighting the benefits of the adopted technologies in terms of increased bandwidth, low latency and high reliability.
ACKNOWLEDGMENT This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 957403. The content of this article reflects the author's view and the Commission is not responsible for any use that may be made of the information it contains.