Implementation and evaluation of frequency division multiplexing of numerologies for 5G new radio in ns-3

The New Radio (NR) access technology that is currently being developed by the 3GPP for fifth generation (5G) systems is expected to be extremely flexible to allow operation in a wide range of bands and address a variety of use cases. One of the key steps towards this objective is the inclusion of a set of numerologies, which are specified by subcarrier spacing and cyclic prefix. Different numerologies can be exploited to meet different quality-of-service (QoS) requirements in terms of latency and throughput. To achieve this, 3GPP specifies that the NR system shall support multiplexing of numerologies in time and/or frequency domain. In this work, we focus on frequency division multiplexing (FDM) of numerologies. To allow simulation of FDM of numerologies in the ns-3 network simulator, we have extended the mmWave module with some of the key features defined for NR systems. These extensions include support for NR frame structure, numerologies, and on top of that the FDM of numerologies. To implement the FDM of numerologies, we leveraged the carrier aggregation (CA) feature of ns-3 LTE module, and for that purpose, we have extended the mmWave module to support the CA functionality. In this paper, we describe these extensions and we carry out a performance evaluation of numerologies and FDM of numerologies.


INTRODUCTION
3rd Generation Partnership Project (3GPP) is devoting significant efforts to define the fifth generation (5G) New Radio (NR) access technology [1], which is expected to be extremely flexible from its physical layer definition and up to the architecture, in order to be able to work in a wide range of bands and address many different use cases. In this regard, one of the key steps forward of NR with respect to 4G Long Term Evolution (LTE), is the inclusion of a flexible orthogonal frequency division multiplexing (OFDM) system by means the numerologies [4]. NR defines a set of numerologies, which specify a subcarrier spacing (SCS) and a cyclic prefix overhead (CP), to handle a wide range of frequencies and deployment options [1]. Also, a base station (a.k.a. gNB in NR) should provide access to different types of services, as enhanced Mobile Broad-Band (eMBB), massive Machine Type Communications (mMTC), and Ultra-Reliable and Low Latency Communications (URLLC). A latency-throughput trade-off appears when attempting the selection of the proper numerology for gNB's operation: larger SCS is better to reduce latency and for complexity (i.e., for URLLC traffic), while lower SCS is better for high throughput performance (i.e., for eMBB traffic). For that reason, 3GPP NR specifies that multiple numerologies should be able to be multiplexed in time and/or frequency for a better user (a.k.a. UE in NR) performance under different types of data [11]. This way, a gNB can accommodate different services with different latency requirements, as eMBB and URLLC, within the same channel bandwidth. When numerologies are multiplexed in a frequency domain, a.k.a. frequency division multiplexing (FDM) of numerologies, each numerology occupies a part of the whole channel bandwidth, which is referred to in NR as a bandwidth part (BWP) [2]. In addition, it is shown in [6] that occasions where multiplexing of numerologies in time domain, a.k.a. time division multiplexing (TDM) of numerologies is needed are rare if the gNB configures properly the FDM of numerologies for eMBB and URLLC traffics. 3GPP agreed that the BWP allocation has to be statically or semi-statically configured [11].
Since the standardization of NR access technology is currently under development, a network simulator that is capable of simulating emerging NR features is of great interest for both, scientific and industrial communities. In recent years a lot effort has been done by New York University (NYU) Wireless and University of Padova to develop a simulator that will allow simulations of communications in millimeter-wave (mmWave) bands, which represent a central technology of future 5G cellular wireless systems, such as NR, public safety and vehicular communications. Hence, a new mmWave simulation tool has been developed as a new module of widely used ns-3 network simulator [9]. ns-3 is an open-source discrete-event network simulator that allows full-stack simulations. A complete description of the mmWave module is provided in [7]. The mmWave module source code is still not part of the standard ns-3 distribution and is available at a different repository [8]. mmWave module implements a complete 3GPP protocol, where the physical (PHY) layer is developed to support a new mmWave-based channel, propagation, beamforming and antenna models; and the medium access control (MAC) layer to support time division duplex (TDD), time division multiple access (TDMA) MAC scheduling, enhanced hybrid automatic repeat request (HARQ) for low latency applications. The higher layers are mostly based on ns-3 LTE module functionalities, thus still following 3GPP LTE specifications, but extending it to involve some of advanced features that are expected to emerge in 5G networks, such as dual connectivity and low latency radio link control (RLC) layer.
In our work, we aim at following NR standardization process and thus developing and simulating NR features that will most probably be included in future 3GPP NR standardization process. Some of the features that are to some extent already defined in [1] are frame structure, numerologies and FDM of numerologies. To create a NR compliant simulation tool, we have modified and extended the mmWave module to support the aforementioned features. To implement the NR frame structure and flexible numerologies we have modified the implementations of PHY and MAC of the mmWave module. On the other hand, to implement the FDM of numerologies, according to our design, the prerequisite is the carrier aggregation (CA) feature, which was recently added to the LTE module [5], but is not yet available in the mmWave module. Thus, to implement the FDM of numerologies, we have extended the mmWave module to support the CA, and on the top of that feature we have built a framework for the configuration and simulation of FDM of numerologies. Thus, the main contributions of the present work are extensions to the mmWave module that support 1) NR frame structure, 2) numerologies, 3) FDM of NR numerologies, and 4) CA extension to the mmWave module. We also provide end-to-end performance evaluations of NR frame structure, numerologies and FDM of numerologies.

NR KEY FEATURES
The NR access technology that is being developed by 3GPP for 5G systems is expected to be extremely flexible to allow operation in a wide range of bands (from sub 1 GHz to 100 GHz), cover multiple deployment options, address different use cases, and operate under multiple spectrum access paradigms (licensed, unlicensed, shared) [1]. In the following subsections, we provide a summary of some key features of NR, to introduce basic nomenclature and concepts required to understand its implementation in ns-3.

NR Frame Structure and Numerologies
With flexibility in mind, NR includes multiple numerologies, being each defined by a SCS and a CP [1]. The numerology µ can take values from 0 to 4, and specifies a SCS of 15x2 µ kHz and a slot length of 1/2 µ ms. The supported SCSs are in the range from 15 to 240 kHz in the current NR specifications, where larger SCSs are to be used at higher carrier frequencies. The number of subcarriers per physical resource block (PRB) is fixed to 12, and the maximum number of PRBs according to Release 15 is 275. The frame length is 10 ms, and a frame is composed of 10 subframes of 1 ms each,  to maintain backward compatibility with LTE. A slot is defined as 14 OFDM symbols. Slot formats have been defined in [2], and they can contain all downlink (DL), all uplink (UL), or at least one DL part and at least one UL part. So, the PRB width is equal to 180x2 µ kHz, and the OFDM symbol length is equal to 1/(14x2 µ ). In Table  1 we show the configuration of different numerologies. Note that µ = 0 corresponds to the LTE system configuration, while µ > 0 enables larger bandwidth and shorter transmission time interval (TTI), which is useful for mmWave bands. The difference with respect to LTE is that, the SCS and the OFDM symbol length can have different values depending on the numerology that is configured. NR also defines mini-slots composed of 2 OFDM symbols up to the slot length -1 in any band, and of 1 symbol, at least above 6 GHz. The purpose of mini-slots is to reduce the latency by providing more flexibility for the transmission of the small amounts of data. Additionally, the concept of self-contained slot is recently proposed as an important feature [10] to be supported to provide a significant latency reduction by reducing a delay in reception of e.g. HARQ feedback or UL grant. For example, ACK/NACK is scheduled in the same slot as DL data, or UL grant is followed by UL transmission in the same slot. Both of these features, mini-slots and self-contained slots, can be further combined with the FDM feature to accommodate different types of traffic and DL-UL traffic asymmetries in an adequate way.

FDM of Numerologies
An additional level of flexibility in NR system can be achieved by employing the multiplexing of numerologies in the frequency domain. In general, URLLC traffic requires a short slot length to meet strict latency requirements, while eMBB use case in general aims at increasing throughput, which is achieved with a large slot length [4]. Therefore, among the set of supported numerologies for a specific operational band and deployment configuration, URLLC shall be served with the numerology that has the shortest slot length, and eMBB with the numerology associated to the largest slot length [11]. That is, the numerology for URLLC will be larger than the numerology for eMBB, µ u > µ e , where µ u is numerology used for URLLC and µ e for eMBB. Hence, the main objective of FDM of numerologies is to address the trade-off between latency and throughput for different types of traffic. In Figure 1 we illustrate an example of FDM of two numerologies. In the example, the channel is split into two BWPs that accommodate the two aforementioned numerologies (µ u and µ e ) multiplexed in frequency domain. The total bandwidth B is divided into BWPs of bandwidth B u for URLLC and B e for eMBB, so that B u + B e ≤ B. The number of PRBs for URLLC is N u and N e for eMBB. Note, that the PRB width varies with the numerology.

NS-MMWAVE MODEL CHANGES
To enable previously described features in the mmWave module we have remodeled the frame structure, defined numerologies and implemented the FDM of the numerologies. In the following subsections, we provide a detailed description of these changes.

NR Frame Structure and Numerologies Implementation
At the time of writing this paper, the latest mmWave module provided by NYU Wireless and University of Padova does not support NR frame structure and a flexible configuration of NR numerologies. Currently, the mmWave module offers an implementation of the frame structure according to a specific design proposed by the authors of the module. A detailed description of the mmWave module frame structure can be found in [7]. The main difference compared to the NR frame structure is that there is no notion of slot as defined per NR specifications, which we explained in Section 2.1. Thus, according to the mmWave module, to modify the numerology one would need to e.g. change the subframe length, which in turn should be fixed to 1 ms according to the NR frame structure, or by changing other parameters. However, without having the slot granularity it is hard to match the NR configurations of the numerologies by modifying the available parameters. The authors of the mmWave module propose a specific frame structure configuration which is similar to the NR µ = 4, but has some important differences. E.g. the subframe, which determines the granularity of the MAC scheduling in the mmWave module, is composed of 24 OFDM symbols, while in NR, the granularity of the MAC scheduling is per slot which is composed of 14 OFDM symbols. Additionally, in the frequency domain the total bandwidth is composed of 72 subbands out of which each subband has bandwidth of 13.89 MHz and is composed of 48 subcarriers, while the NR structure assumes a fixed number of SCSs per PRB, and this number is 12. Since the number of subcarriers per PRB is fixed, the SCS defines the size of PRB, and also the total number of PRBs of the NR system. In the case of µ = 4, the width of the PRB should be 2.88 MHz, and the total number of PRBs depends on the system bandwidth.
To support the NR frame structure in the time domain and the scheduling operation per slot granularity, we have introduced an additional granularity level in time domain, the slot. Thus, according to the new frame structure design, frame has length of 10 ms and is  split into 10 subframes, each of duration of 1 ms. Each subframe is further split in time into a variable number of slots that depends on the configured numerology. The number of OFDM symbols per slot is 14 OFDM symbols as specified in [1]. In Figure 2 we illustrate the NR frame structure in time domain when configured for µ = 4. Figure 3 shows the NR frame structure in frequency domain. For µ = 4, SCS = 240 kHz and Y = 2.88 MHz. Assuming a total channel bandwidth of 400 MHz, then K=138 and X=397.44 MHz. In order to simplify modeling, the CP is included in the OFDM symbol length. For example, for µ = 4, Z = 62.5/14 = 4.46 us, which accounts for the real OFDM symbol length 1/SCS = 4.17 us and a CP of 0.29 us.
The PHY transmission and reception and the MAC scheduling are adapted to the new NR frame structure. In the mmWave module, the functions StartSubFrame() and EndSubFrame() of the MmWaveEnbPhy and MmWaveUePhy classes, are responsible for handling the transmission and the reception of the subframes. According to our model, since the transmission and the reception are per slot level, latter functions are replaced by StartSlot() and EndSlot(), respectively. These functions are executed with a fixed periodicity, i.e., every 14 OFDM symbols, but the real duration of the slot depends on the configured numerology. Please note that the mmWave module also contains functions called StartSlot() and EndSlot() which are used for transmission and reception of TTIs of variable length, and which are in the mmWave module referred to as slots. In our model instead, the corresponding functions are called StartVarTti() and EndVarTti(), respectively. These functions correspond to TTI as defined in [1]. According to the NR definition of TTI, one TTI duration corresponds to a number of consecutive symbols in the time domain in one transmission direction, and different TTI durations can be defined when using different number of symbols (e.g., corresponding to a mini-slot, one slot or several slots in one transmission direction). Thus, TTI is in general of variable length regardless numerology. To support operation per slot, instead of SfAllocInfo structure that is used in the mmWave module to hold the information corresponding to a subframe, we introduce SfAllocInfo that holds the information per slot, and it extends the SfAllocInfo structure according the NR frame structure. One SfAllocInfo is built of a list of VarTtiAllocInfo elements. VarTtiAllocInfo contains the scheduling information for a single TTI whose duration is no longer than that of one slot. At the UE PHY, VarTtiAllocInfo objects are populated after reception of Downlink Control Information (DCI) messages. For each VarTtiAllocInfo the scheduler specifies which are containing contiguous OFDM symbols, along with the information whether the allocation is DL or UL, and whether is control or data. Currently, according to the mmWave module design, and in accordance with the self-contained slot structure in NR, the first and the last OFDM symbol of the frame structure are reserved for, respectively, DL control and UL control. We configure the MAC-to-PHY processing delay at the gNB to be numerology-dependent and is configured to be a specific number of slots. It defaults to 2 slots. On the other hand, the transport block decoding time at UE is by default fixed and equals to 100 us, but it could also be easily configured to be numerology-dependent.
Differently from the LTE and the mmWave module design in which the MAC scheduling is performed per subframe, according to the new NR frame structure design it is performed per slot. A slot indication to the MAC layer triggers the scheduler at the beginning of each slot to allocate a future slot. The mmWave module MAC layer currently supports only TDMA scheduling. The existing TDMA schedulers provided by the mmWave module have been enhanced to support the new NR frame structure by means of operating at a slot level, instead of the subframe. These schedulers are: MmWaveFlexTtiMacScheduler, MmWaveFlexTtiPfMacScheduler, MmWaveFlexTtiMaxRateMacScheduler and MmWaveFlexTtiMax-WeightMacScheduler.
According to our design the mmWave module allows a flexible configuration of numerologies. The numerology is specified by the numerology attribute of MmWavePhyMacCommon class. Once the numerology is configured, the lengths of the symbol, the slot, the SCS, and the number of PRBs are dynamically determined in a runtime. Our implementation currently supports numerologies shown in Table 1. It also support µ = 5 which might be used in future NR releases for operation in high carrier frequencies. µ = 5 is defined by SCS of 480 kHz, OFDM symbol length of 2.08 us, CP of 0.15 us, slot length of 31.25 us, and there are 32 slots in a single subframe. Furthermore, additional µ values could be easily configured thanks to the developed generic implementation. For example, 3GPP does not define the control plane functionality of the FDM. The PHY needs a capability of configuration of different BWPs, and this configuration may be static or semi-static. On the other hand, from the simulation point of view, a typical use case scenario for FDM of numerologies would be to allow configuration of a small number of different BWPs, and as a preliminary implementation of this feature it would be sufficient that this configuration is static. At this point, this is an important design assumption for two reasons. Firstly, the current implementation of the channel and the propagation loss model of the mmWave module is not designed in a way to allow runtime modifications of the physical configuration parameters related to frequency, such as: the system bandwidth, the central carrier frequency, and neither time related physical layer parameters, such as symbol and slot length. Thus, until the current 3GPP mmWave channel model is not being modified to allow these runtime configuration changes, it will not be possible to perform semi-static reconfiguration of BWPs. Additionally, since the control plane functionality of the FDM is not defined yet by 3GPP and it is not clear how the runtime reconfiguration would be performed, i.e., the PHY reconfiguration of BWPs, the periodicity, and the control messages. Some options that are being considered by 3GPP are via the broadcast channel, RRC signaling or periodical group common PDCCH.

FDM of Numerologies Implementation
Regarding the data plane, note that there is a similarity of the concept of the component carriers (CCs) in LTE and the BWPs in NR. While the purpose is different, the concept of having aggregated physical layer resources remains the same. The main difference is that in LTE, the different carriers have the same symbol, slot, subframe, and frame boundary, while in NR only the subframe and frame boundaries of different BWPs are aligned. Another difference is that the bandwidth of a CC in LTE is predefined and cannot be dynamically changed. On the other hand, in NR it is expected to allow semi-dynamic reconfiguration of the bandwidth of different BWPs. Additionally, in NR the SCS may be BWP-dependent, while in LTE it is the same for all sub-carriers. However, in the case of static BWPs configuration, possibility of having different configurations of time and frequency among BWPs does not make the BWP concept in its essence very different from the CCs in LTE.
Taking into account the previously explained assumptions along with the currently available requirements for the control and data plane implementation of FDM of numerologies, the main design idea is to leverage the CA feature of LTE module to implement the basic FDM feature in mmWave module. Note that the CA feature is mainly seated at the RRC and MAC layers. Since the current implementation of the mmWave leverages the LTE module implementation of RRC, incorporating this part of CA functionality is automatically done by upgrading the LTE module to the version that includes the CA feature. In this way, the mmWave module inherits the features, such as extension of RRC messages in order to handle the secondary CC (SCC) information. By leveraging LTE CA RRC functionality, each NR UE establishes the cell connection following the LTE R8 procedure. This includes a cell search and selection, followed by the system information acquisition phase and the random access procedure. The RRC is in charge of the connection procedure, and it configures the primary CC (PCC). Once the UE is in connected state, the gNB RRC is responsible to perform reconfiguration, addition and removal of SCCs. Since the UE Capability Inquiry and UE Capability Information are not implemented it is assumed that each UE can support any BWP configuration provided by the gNB to which it is attached. Since, 3GPP defines PCC as UE related and not as eNB related, according to LTE CA the PCC is the CC that is perceived as the most robust connection for each UE.
While, the RRC CA functionality is inherited from LTE module, the rest of the protocol stack of both, gNB and UE, needs to be updated according the CA architecture. This means that the models of both MmWaveEnbDevice and MmWaveUeDevice shall be extended to support the installation of the MAC and PHY per carriers. Additionally, the MAC layer of mmWave module needs to be updated to allow scheduling on different BWPs. It should be defined how the MAC scheduling will be performed in multi-BWP system, and how will be this information transmitted to the UE. Different options are proposed recently in [11] to be included in 3GPP specification: (1) dedicated control channel (2) anchor control channel, and (3) shared control channel.
Sending the scheduling information through a dedicated control channel of each BWP assumes that the UE is capable of decoding several control channels simultaneously. Depending on the type of traffic, the UE can be scheduled on different BWPs. This option is analog to independent carrier scheduling in LTE Release 10, according to which each CC has its own physical DL control channel (PDCCH) and the scheduler allocated traffic per CC-basis. Anchored control channel is transmitted on the BWP that is always ON or been used most frequently. Transmitting the scheduling information through anchored control channel means that the control information for all BWPs is transmitted on the same BWP. This is similar to cross-carrier scheduling in LTE Release 10, according to which the scheduling information is sent through the PCC. The third option, shared control channel, means that the control channel spans over BWPs in which the UE is being scheduled. Thus the bandwidth of the shared control channel dynamically changes depending on the scheduling information for UE.
In the current implementation of FDM of numerologies in ns-3, we support the transmission of the scheduling information through a dedicated control channel in each BWP, and the MAC scheduling and HARQ processes are performed per BWP. Finally, according to our model, the multiplexing of the data flows based on the type of traffic is performed by a new entity called BWP manager, whose role is similar to that of CC manager in the LTE module. The BWP manager can use 5G quality-of-service (QoS) identifiers (5QI), as defined in [3], to determine on which numerology (i.e., BWP) to allocate the packets of a radio bearer and to establish priorities among radio bearers.
3.2.2 Implementation of FDM of Numerologies. The current implementation of the mmWave module repository does not include the latest release of the ns-3 LTE module and the CA feature. Thus, as a first step, we merged the LTE module code of the mmWave module with the latest ns-3-dev LTE module that includes the CA feature. However, upgrading of the LTE module to the latest ns-3-dev version is not enough for the correct functioning of CA with mmWave module, since the latest version of the LTE module expects the service access point (SAP) interfaces of the lower layers and the stack initialization according the CA design. The class that is responsible for these initializations in the mmWave module is MmWaveHelper class. Thus, the second major changes in the mmWave module to enable CA architecture are in the functions of MmWaveHelper class for installation of gNB and UE, and these are respectively, InstallSingleEnbDevice and InstallSingleUeDevice. Additionally, the function that has suffered important changes in MmWaveHelper is DoInitialize which is responsible for the initialization of the channel and the pathloss model.
InstallSingleEnbDevice function is modified to allow configuration of the gNB according to CA architecture. In Figure 4 we illustrate the changes in the model of gNB device, which are highlighted in dark green. The gNB device which is in mmWave module represented by MmWaveEnbNetDevice is extended to include the map of CCs. Similarly to the LTE module, a CC consists of the MAC and PHY layers of each CC. Since the MAC and PHY configuration parameters in MmWavePhyMacCommon instance are extended to support a flexible numerology configuration, each CC can now be configured as a separate BWP with different numerology and other BWP specific parameters: carrier frequency, bandwidth, L1L2 processing and decoding delay, HARQ timing, etc. Apart from latter parameters, another important BWP-specific parameter is the transmission power per BWP which can be configured through the attribute of MmWaveEnbPhy. MmWaveEnbNetDevice is extended to include the CC manager entity whose interface is implemented in abstract class called LteEnbComponentCarrierManager. Already existing set of LTE CC manager implementations is extended with a new CC manager entity called BwpManager, which is different from LTE CC manager since it can make decisions based on the numerology configuration of carriers. The role of BwpManager class is to allow differentiation of the traffic flows based on their QoS requirements, and to forward the flow to the BWP which would provide the minimum latency for latency-sensitive types of flows, and which would maximize the throughput for the traffic flows that do not have a tight latency constraint. Current implementation of BwpManager supports all LTE EPS bearer QoS types, and the assignment of the corresponding BWP is based on the static configuration provided by the user. I.e., user shall specify which QoS type shall be mapped to which numerology and BWP. Similarly to the changes in gNB device that we illustrated in Figure 4, we have also extended the NR UE model to support the CCs and UE CC manager. However, in the current implementation, we are interested only in FDM of numerologies coordinated by the gNB and its BWP manager. Additionally, the function InstallSingleEnbDevice of MmWaveHelper is responsible for building up the whole protocol stack of the gNB and installing the SAP interfaces for control and data planes. In Figure 5 we illustrate the gNB data plane according to this implementation. The main changes are installation of CC manager between the RLC and the MAC layers, and installation of CCs. To allow this architecture, the CC manager implements the SAP interfaces that were previously handled by RLC and MAC. In this way, gNB MAC and PHY continue to function as in the previous architecture, while CC manager is in charge of handling the information needed for CC management and multiplexing the flows over different carriers (or BWPs). Also, the control plane architecture of the gNB device is updated according the LTE module CA architecture. The full description of the LTE module CA architecture is provided in [5]. Similarly, the function InstallSingleUeDevice is modified to allow installation of NR UE device according to the LTE module CA architecture. InstallSingleUeDevice builds the CCs, installs the CC manager, and builds the protocol stack that is analog to the LTE UE CA. DoInitialize function of MmWaveHelper is modified to allow configuration of different BWPs. However, this initialization is different from DoInitialize of LteHelper due to the current limitation of the mmWave module channel models (MmWaveBeamforming, MmWaveChannelMatrix, MmWaveChan-nelRaytracing, MmWave3gppChannel), all of which depend on the various PHY and MAC configuration parameters specified through a single instance of MmWavePhyMacCommon. Hence, due to the current design of the mmWave module and channel models it is not possible like in LTE module to install devices operating on different central carrier frequencies, neither to install different CCs. Thus, to enable installation of different CCs, we have modified DoInitialize to allow installation of SpectrumChannel instance per CC. In this way, a single simulation will have as many MmWave3gppChannel channel model instances as there are BWPs configured. However, an important limitation of this design is that all gNBs and UEs in the simulation have the same BWP configuration. Each MmWave3gppChannel instance will represent a channel model for a different BWP whose parameters are defined in a corresponding instance of MmWavePhyMacCommon class. In this section, we explain how the developed features can be used in simulations and we provide end-to-end performance evaluations of different configurations of numerologies and BWPs. Here, we present the results of the simulation implemented in cttc-3gppchannel-nums-fdm.cc script. This program can be configured to evaluate either different NR numerologies or FDM of numerologies by using 2 BWPs. In the following subsections, we describe each of these versions of scenario, and we provide the corresponding simulation results.

Evaluation of Numerologies
In the first scenario, for a single BWP, we evaluate different numerologies while varying the traffic load. The numerologies that we evaluate in this simulation are µ = 1, ..., 5. The traffic is DL UDP. Packets of 100 bytes are being generated with λ rate (in packets/s) which takes the following values: 1250, 62500, 125000, 250000, 375000. The simulation scenario is composed of a single gNB and a single UE that are at 10 m distance. The height of the gNB is 10 m and of UE is 1.5 m. The gNB transmission power is 1 dBm, the central carrier frequency is 28.1 GHz, and the bandwidth is 100 MHz. The channel model is the 3GPP channel model, and the scenario is UMi-StreetCanyon. Beam search is performed by using the long-term covariance matrix method [7]. In Figures 6 and 7, we show the mean latency and the mean throughput of the flow versus the offered load, when the system is being configured to use different numerologies. From Figure 6, we observe that when the system is not saturated there is a significant impact of the numerology on the delay. However, when it is saturated there is a great impact of higher layers buffer delays and the actual transmission time does not play an important role. However, with respect to throughput, we observe a different effect in Figure  7. When there is no saturation, each numerology is able to satisfy QoS requirements of the flow equally, i.e. the served rate is equal to the offered rate. But, once in the saturation region, the lower numerologies (1, 2 and 3) start to provide a slightly better throughput than the higher numerologies (4 and 5). The reason for this is that depending on the bandwidth and SCS of each numerology a different portion of the bandwidth will be utilized. E.g., for µ = 5, the SCS is 480 kHz, PRB width is 5.76 MHz, thus the actual bandwidth used is 97.92 MHz, while for µ = 1, the SCS is 30 kHz, PRB width is 0.36 MHz, thus the actual bandwidth is 99.72 MHz. Hence, there is around 2% more bandwidth in the second case, which corresponds to the difference in the achieved throughput that we see in the Figure 7. Note that the difference in the maximum throughput achieved by different numerologies depends on the channel bandwidth, and it is more remarkable for low bandwidths.

Evaluation of FDM of Numerologies
In the second simulation configuration, we evaluate the performance of FDM of numerologies when the total system bandwidth is divided in 2 BWPs of equal size. There are two BWPs, of 100 MHz bandwidth each. The total transmission power is 4 dBm, which is uniformly distributed among the two BWPs, so that each BWP disposes of 1 dBm. The central carrier frequencies are 28.1 GHz and 28 GHz. First BWP is configured with µ = 4 and the second BWP with µ = 2. The topology consists of 1 gNB and 2 UEs. The first UE requests URLLC DL flow, and the second UE demands of an eMBB DL flow. URLLC flow packets are of 100 bytes and they are transmitted with λ from 1250, 62500, 125000, 250000, 375000. eMBB flow packets are 1252 bytes and they are transmitted with 0.1 × λ rate. Packet sizes and λ are configured in this way to achieve the same offered throughput for both flows for any value of λ. The packet sizes are adjusted to account for the UDP header of 28 bytes.
UEs are placed at 10 m distance from gNB. When there are 2 BWPs, the BWP manager is configured to forward URLLC traffic over the BWP with a higher µ, and eMBB over the BWP with a lower µ. We compare the performance of 2 BWPs configuration with a single BWP, of the same total bandwidth (200 MHz) and total power (4 dBm), and µ = 2.
In Figures 8 and 9 we show the performance in terms of the mean delay and the throughput versus the offered load per flow. As expected, we observe a positive impact of FDM of numerologies on the mean delay of URLLC flow. This flow benefits from being scheduled via BWP of a higher µ. On the other hand, eMBB flow has almost the same performance in the terms of delay regardless if the FDM of numerologies is being used. With respect to throughput, we observe that URLLC flow obtains a better performance without FDM and when is transmitted over lower µ in the saturation regime, due to the reasoning given in the previous subsection. However, there is almost no impact of FDM of numerologies on the performance of the eMBB flow.

CONCLUSIONS
In this paper, we presented several extensions to the mmWave module to support some of the key features of the NR systems along with the CA feature according to the LTE Release 10 specification. We introduced a new time level, the slot, which defines the MAC scheduling and PHY transmission/reception unit. We automatically configured the time/frequency resource units as a function of the numerology and channel bandwidth, by following the 3GPP specifications. Finally, we implemented FDM of numerologies, leveraging on the CA feature of LTE, and for which we defined a BWP manager. Finally, we carried out the performance evaluation of NR numerologies and FDM of numerologies. This enables evaluation of the delay/throughput trade-off from an end-to-end perspective.
As part of the future work, we consider that several features require attention and shall be included into the mmWave module to allow simulations of different use cases for NR systems. As we mentioned in FDM of numerologies implementation subsection, Section 3.2.2, we consider that would be important to allow different configurations of PHY and MAC parameters related to the BWP allocation for gNBs and UEs. Once that this functionality is implemented, it shall be allowed to have different BWP configurations for different gNB and UEs. Additionally, an interesting future work is the definition of a procedure for semi-static configuration of the BWP allocation that could take long-term system statistics into account to properly distribute the spectrum. Finally, some more sophisticated BWP manager could be implemented, i.e. one that would perform the BWP optimization not only based on the QoS flow parameters but also using real-time network measurements, such as long-term traffic statistics. Finally, on top of the FDM of numerologies, potential future work includes the development of a framework to allow dynamic TDM of numerologies by means of puncturing, which would be useful e.g. in case of a sudden need of a large bandwidth for URLLC. In this case, puncturing the resources already devoted for eMBB and defining procedures to indicate so through an indicator of preemption to recover the data would be required.