A Pub/Sub SDN-Integrated Framework for IoT Traffic Orchestration

The Internet of Things (IoT) is advancing and the adoption of internet-connected devices in everyday use is constantly growing. This increase not only affects the traffic from other sources in the network, but also the communication quality requirements, like Quality of Service (QoS), for the IoT devices and applications. With the rise of dynamic network management and dynamic network programming technologies like Software-Defined Networking (SDN), traffic management and communication quality requirements can be tailored to fit niche use cases and characteristics. We propose a publish/subscribe QoS-aware framework (PSIoT-SDN) that orchestrates IoT traffic and mediates the allocation of network resources between IoT data aggregators and pub/sub consumers. The PSIoT framework allows edge-level QoS control using the features of publish/ subscribe orchestrator at IoT aggregators and, in addition, allows network-level QoS control by incorporating SDN features coupled with a bandwidth allocation model for networkwide IoT traffic management. The integration of the framework with SDN allows it to dynamically react to bandwidth sharing enabled by the SDN controller, resulting in better bandwidth distribution and higher link utilization for IoT traffic.


INTRODUCTION
network-wide specifications, application domain and IoT characteristics. The PSIoT-SDN framework expands on the usual orchestrator's edge-level QoS control operation by using SDN network programmability coupled with a bandwidth allocation model (BAM). We evaluate the enhancements in QoS when integrating the framework with the SDN network manager and how it positively affects IoT traffic QoS over the network. As such, contribution of the PSIoT-SDN framework will be to integrate a QoS approach on network edge with a network bandwidth sharing strategy based on BAM models that is SDN-controlled.
In this article we'll describe the PSIoT-SDN framework and how it proposes to manage IoT traffic, as well as how it combines with SDN to offer enhanced QoS capabilities to IoT traffic. We present in section 2 previous work in that handle QoS and traffic management. In section 3 we explore proposed IoT-oriented architectures, considering both data processing and QoS in an SDN enabled network. Section 4 overviews PSIoT-SDN framework components including its basic orchestrator for IoT traffic management at network edgelevel (aggregators) and the SDN controller for network-wide QoS management. In Section 6 we present a proof of concept for the PSIoT-SDN with a SDN network controller coupled with a bandwidth allocation model (BAM) and evaluate how it affects the traffic in the network. Finally, section 7 concludes with an overview of the result and what was presented.

RELATED WORK
IoT applications and devices are varied in communication protocols, hardware and characteristics. This has generated a large amount of IoT-focused works in the literature ranging from hardware and network protocols to applications and traffic protocols and management [7].
A comprehensive survey in [3] reviews the enabling technologies, protocols and applications for IoT, presenting elements required for devices to deliver IoT functionality. The work in [10] presents the IoT key features and driver technologies, detailing IoT into application domains like industry, smart city and health.
Considering IoT traffic in wireless networks, [4], [31], [2] propose traffic management solutions that focus on wireless sensor networks (WSN) and wireless cellphone networks. The research focus is mainly in considering the constraints of low power and computation on IoT devices and wireless networks.
The research categorized in the IoT-SDN integrated approach are mainly focused on methods to manage the IoT traffic being generated by IoT devices after it has entered the network. These are mostly about using the SDN dynamic programming capability and Cloud-related technology to offer better IoT traffic control towards mobility, security, spectrum and service management [32] [37] [11] [13].
Using the Fog for edge data processing and aggregation, [14] and [29] present platforms for orchestrating Fog workload. The research in [20] extends Fog computing to improve network resilience in Fog nodes and [38] does an extensive survey on edge computing for the IoT.
The research in [36] presents a publish/subscribe communication platform for IoT services. It is based in SDN and directly uses the network's SDN management, and knowledge on the network's routing and topology, to control the QoS of topic subscriptions traffic.

IOT ARCHITECTURE, DATA PROCESSING AND SDN INTEGRATION
The Internet of Things is growing more and more as users and applications arise with IoT connected devices. The IoT enables everyday objects to be identified and to communicate with each other and other applications over the internet. This flexibility also means that the IoT is extremely heterogeneous, in both data and network usage. As the IoT expands, effort has been made to find common traits and characteristics of the IoT connected devices as well as architectures for IoT data processing and network communication.

IoT Architecture
The heterogeneous nature of IoT devices, data and overlapping application domains makes it difficult to enable interoperability between IoT devices and applications, despite IoT traffic characteristics and requirements being well defined.
In [12], the ETSI Technical Committee for Machine-to-Machine Communications (ETSI TC M2M) proposed an IP-based architecture leveraging existing technologies. The architecture was divided into three domains: • The Application Domain, where client and M2M applications lie; • The Network Domain, the network between applications and device gateways; and • The Device & Gateway domain, the location the devices and/or gateways reside.
The definition of these domains are useful in organizing and designing IoT solutions, as it provides a clearer picture on how IoT devices and data happen in networks and the internet. This clearer distinction can allow for tailored solutions, catering to each domain specifically and allow a better integration between architectures and IoT solutions, like data processing or IoT QoS.

IoT Data Processing
The growing number of IoT devices generates an also growing amount of data being transmitted through the internet and wireless networks. This amount of traffic can burden the network and cause IoT services that rely on timely communication to malfunction. Aggregating and preprocessing IoT data can alleviate the burden on the network. This process can happen at two points: in-network (edge and network devices) and at the cloud level [1] [35]. Cloud IoT data processing aims to cut down on IoT traffic to IoT data consumers by processing device data into aggregated data of interest to many consumers.
Further data processing also happens in local IoT device networks, such as wireless, and wireless mesh networks. This occurs because of the energy and resource constraints on these devices, where neighbouring data that is usually highly redundant is aggregated, and reduced in size, thus preserving energy and resources.
Data aggregation is responsible for increasing the network lifetime and reducing the energy consumption [33]. A comparison of data aggregation techniques is presented in [26], and [23] presents a literature survey on data aggregation mechanisms.
Cloud aggregation and data processing don't fully supply the IoT need for low latency, leading to the search of some derived solutions. Considering this problem, Cisco introduced in [8] the concept of Fog Computing to enable applications on billions of connected devices already connected in the Internet of Things to run directly in the edge of the network, providing mobility and geo-distribution support, as well as managing location and latency.
A hybrid approach to data aggregation exists, where Fog nodes act as preprocessors that aggregate IoT data in the network edge before forwarding it into the Cloud, presented in [5].
Considering these data aggregation methods, and the ETSI IoT domain model, Figure 1 shows how the different IoT data aggregation are located in the network. It shows how IoT data flows from devices in the M2M Domain, is aggregated and preprocessed in the Fog and forwarded to the Cloud. This stream of aggregation and processing can cut back on the load that massive IoT traffic can cause in the network.

IoT Network Architecture with SDN
With more complex networks, traditional management paradigms must be reworked and changed. There is a need for network management approaches that are scalable and are able to work well with the heterogeneous, complex, and large-scale nature of IoT.
The great number of heterogeneous IoT devices and data require new technology and management patterns to better serve IoT traffic QoS needs. SDN is a well-matched network programming paradigm suitable for IoT management, that separates the control and data planes. SDN provides high-level abstraction and virtualized network functions that make management simpler [34].
SDN-based IoT management allows for better control over traffic routing and network configuration, allowing the network to more easily react to network traffic needs. The dynamic configuration capabilities of SDN networks are a great match to the heterogeneity of IoT traffic, allowing the network to better route IoT traffic and manage its impact on the whole network.
Integrating IoT frameworks with SDN gives greater control over IoT traffic. Figure 2 shows a high-level view of a a typical architecture for IoT-SDN integration. The SDN-IoT integration brings several significant benefits for IoT traffic: (1) Intelligent traffic routing and better network resources use.
(2) Simplified information acquisition facilitating information analysis, decision making and network configuration actions. (3) Virtualization, whenever required, may be easily achieved and deployed using common SDN virtualization tools like hypervisors [15]. (4) Visibility of network resources and access management based on user, group, device, and application. (5) Intelligent algorithms to build effective traffic pattern analyzers. These benefits result in IoT networks with integrated SDN capabilities becoming more agile, scalable and based on demand.

THE PSIOT FRAMEWORK BASICS
As described in [21], the PSIoT-Orch is a IoT traffic management framework that combines the Cloud, Fog and Pub/Sub IoT scenarios to enable Pub/Sub-based IoT data transfers and QoS requirements enforcement based on IoT data sets.
The PSIoT-Orch framework was created to manage massive traffic generated by the ever growing number of IoT devices. Its goal is to use Publish/ Subscribe to allow IoT data transfer among producers and consumers and, concomitantly, to handle network resources efficiently according to IoT QoS traffic requirements at edge level.
It adopts a static priority allocation for IoT traffic data classes at aggregators [21].
PSIoT-Orch manages the IoT network resources by leveraging its control on IoT traffic transmission through the network. This management is specially tailored to IoT data and traffic characteristics, enabling better QoS than data-agnostic QoS rules in the network.
Maintaining IoT data aggregators at the edges of the network enables the framework to have more precise knowledge on IoT data, allows pre-processing and all Fog-related benefits while enabling traffic shaping and prioritizing based on the higher priority of IoT data flows.

The PSIoT-Orch architecture
The PSIoT-Orch framework has 3 main components: • A traffic orchestrator; • Fog-like IoT gateway data aggregators (IoTGW-Ag) acting as Pub/Sub publishers; and • Pub/Sub clients, be they end-user applications or Cloud processing centers. Figure 3 shows how the components are distributed in the network. The communication happens by trading messages through the network, with a centralized orchestrator and multiple clients and producers. The IoTGW-Ag are Fog-like nodes that act as IoT data aggregators and Pub/Sub producers. These nodes can take other IoT-related responsibilities such as data pre-processing, backup or caching. Each IoTGW-Ag connects to the PSIoT orchestrator, that manages the traffic transmission from the many aggregators in the network.
The orchestrator can leverage its control on each IoTGW-Ag in the network to manage the flow of IoT traffic. Figure 4 illustrates how the framework is positioned in the network and how it can leverage the flow of IoT data by managing the traffic flow constraints at each IoTGW-Ag in the network, according to IoT data characteristics.
The orchestrator has the knowledge of each IoTGW-Ag's Pub/Sub subscriptions, as well as the QoS levels required for each topic subscription. This allows it to decide the transmission rates of IoT data from each aggregator in the network, so as to maximize the throughput of higher level QoS subscriptions [21].
The PSIoT-Orch framework relies on the time-sensitivity of IoT traffic to determine its QoS levels. While managing aggregators transmission rates doesn't increase the overall throughput of IoT traffic, it does do so to higher priority IoT topic-based subscriptions. Higher priority IoT data can flow faster in comparison to overall IoT traffic load on the network.
In summary, PSIoT-Orch manages QoS at the aggregators on behalf of IoT traffic by managing the traffic in queues with the orchestrator (Figure 4). In the PSIoT-Orch architecture QoS level assurance is only suported at edge-level at the aggregators. This approach is suitable for any network infrastructure, including the Internet, where no control exists over the network resources being used.

THE PSIOT-SDN ARCHITECTURE
The PSIoT-SDN framework introduces a network-level QoS control in the PSIoT framework. The target now is to achieve both edgelevel and network-level QoS management by leveraging the QoS mangement capabilities at aggregattors provided by the PSIoT orchestrator and to introduce a SDN-based network links bandwidth allocation strategy. For achieving that, the SDN management is coupled with a bandwidth allocation approach for network links resource allocation.
The components of the PSIoT-SDN architecture are shown in Figure 5. In effect, an SDN controller is introduced in the architecture to allow IoT flows programmability at the switches deployed in the network.
In the PSIoT-SDN architecture it is assumed that, firstly, the IoT traffic is heterogeneous in relation to its QoS requirements and is present in multiple network locations. From these highly distributed locations IoT traffic has to travel from IoT sensors to the aggregators and, from there, using a controlled network to its consumers. These factors make it so that finding a common hierarchy of QoS needs for IoT traffic harder and usually leads to tailoring QoS requirements according to local device and network resources capability.
The PSIoT-SDN framework is built to work on top of a controlled network that it can program directly. It is assumed that the framework has direct control over network link resources or routing decisions using the SDN deployed interface. This makes it so that the IoT QoS management has effectively two components: • An IoT QoS traffic management, enabled by the control of all IoT data producers at the IoTGW-Ag queues; and • A link resource management by adjusting bandwidth between network switches all over the path between aggregators and consumers using SDN. SDN enables dynamic network configuration, routing and QoS in networks, and can be regarded as an ally when dealing with IoT traffic. Integrating SDN enabled network management into IoT systems can help to fine tune and fulfill IoT QoS requirements.
Integrating SDN network management within the PSIoT-SDN framework enables enhanced QoS with better guarantees on packet QoS due to the fact that link bandwidth will efficiently allocated among network users.
The next section presents the operation flow of a network running the PSIoT-SDN framework with SDN network management and introduces the bandwidth allocation strategy for IoT flows and other traffic in network links

PSIoT-SDN link bandwidth management and operation flow
The SDN controller functional blocks are illustrated in Figure 6: • An interface with the PSIoT orchestrator; • A MAM-like bandwidth sharing module (MAM -Maximum Allocation Model); and • An SDN/OpenFlow network-programming interface. The SDN controller interface communicates with the PSIoT orchestrator and is responsible for interpreting commands coming The MAM-like module is a bandwidth sharing schema that uses the bandwidth allocation model strategy defined for the MAM model. Bandwidth sharing is an often used concept explored in Bandwidth Allocation Models (BAM) that distributes and efficiently manages scarce network resources [27]. The MAM-like module basically keeps track over the used bandwidth for all links over the path between IoT producer and IoT consumer.
The MAM BAM model assumes that the link bandwidth is divided without any sharing between a set of traffic classes []. For each traffic class (TC) is allocated a slice of the bandwidth and network users are allocated and allowed to use only the bandwidth allocated for its class. This limit is denominated bandwidth constraint (BC).
In the PSIoT-SDN framework the traffic classes and bandwidth constraints were defined and configured as follows: • Traffic class 0 (TC0) supports all low priority traffic, including from the IoT, in the network; • Traffic class 1 (TC1) supports some mid-priority non-IoT traffic; and • Traffic class 2 (TC2) supports high priority traffic other than IoT over the network.
The bandwidth allocated for each traffic class is a management decision configured as BC0, BC1 and BC2.
The SDN controller network-programming interface configures the switches to establish paths for IoT traffic over the network. Paths between producers and consumers are previously calculated using any routing algorithm. The SDN controller uses this computed paths to configure the switch flow-tables over the path.
The SDN controller monitors the network bandwidth usage and network-level IoT class bandwidth allocations. As the traffic changes, the controller can allocate additional bandwidth to IoT traffic by reassigning the TC of IoT traffic flows to TC's with available bandwidth in the links. This is done by the PSIoT-Orch communicating grouped traffic flows, in the form of origin-destination pairs, to the MAM-like module at the SDN controller. This allows the IoT traffic growing on demand to maintain appropriate throughput, by allowing the PSIoT-SDN to utilize the additional bandwidth allocated by the controller to increase the transmission rates of IoT data, according to their QoS need.
The SDN controller operation flow is illustrated in Figure 7. The PSIoT-SDN framework assumes that all traffic flowing in the network are queued at the switches and have bandwidth constraints defined. In effect, the queue implementation and corresponding bandwidth limits are a feature that has dependency of the switched and network operating system used. This is a limitation of the proposed framework but is essential to have bandwidth limits to guarantee QoS requirements at network level [16].

PSIOT-SDN PROOF OF CONCEPT
An implementation of the PSIoT-SDN framework with integrated SDN is made to be easily tested in a simulated environment.
The PSIoT-SDN proof of concept components are: • A network topology created with the Mininet emulator [18]; • The POX [19] SDN network operating system supporting the IoT-SDN network management; • Open vSwitch switches [22], controlled by the OpenFlow protocol; • User traffic generators; • The MAM-like BAM module implemented with OpenStack tools; and • The SDN user-controller software. The proof of concept scenario presents a simple and controlled network topology using links with constrained resources. Its topology and network resources are set in such a way as to present both IoT and normal traffic competing for resources. Figure 7 shows the proof of concept topology with the PSIoT-SDN components on the network. The proof of concept traffic configuration adopted is as follows: • The network link resource allocation is set to 3 traffic classes (TC0, TC1 and TC2) • TC0 is configured initially with 50% of link bandwidth (BC0); • TC1 is configured initially with 30% (BC1); • TC2 is configured initially with 20% (BC2); • IoT traffic is allocated at TC0; and • All other traffic is allocated at TC1 and TC2 according with their priority. The MiniNet emulated OpenFlow switch (Open vSwicth) uses bandwidth controlled network queues (NQ0, NQ1 and NQ2), each of then supporting a traffic class (TC0, TC1 and TC2) [17]. The link's available bandwidth is restrained for traffic flows belonging to each class level. This division is done to simulate real world network constraints on traffic, according to connection SLA's, and IoT traffic is assigned to the lowest priority class to simulate consumer-level internet connections.
This scenario is designed to match common network configurations and to display the better QoS control that the PSIoT-SDN framework can achieve when integrated into an SDN network's management. This will be accomplished by running an implementation of the PSIoT-SDN, which preserves the PSIoT-Orch queue management control at the aggregators, inside an emulated SDN network in Mininet, along with a custom SDN controller that will communicate with the Orchestrator.

Proof of concept simulated runs
This proof of concept simulation aims to show how integrating with the SDN controller has a positive effect on the PSIoT-SDN's network use and on its ability to provide better QoS guarantees to its users.
To this effect, the SDN integration implements this in a way where PSIoT-SDN traffic groups, as signaled to the SDN controller, can be assigned to TC's with spare capacity. Figure 8 shows how the SDN controller reassigns flows of traffic from the PSIoT-SDN to TC's with available bandwidth.
The above network scenario is simulated with a series of events that aim to show the strengths of SDN-enabled bandwidth sharing with the integrating of the SDN network management to the PSIoT-SDN.
The events are ordered as follows: (1) T1 sends traffic to T2. This traffic is in the TC0 class.
(2) C1 subscribes to Ag1 and Ag2. All IoT traffic is in the TC0 class.

Evaluating the results
BAMs allow better network utilization by sharing unused bandwidth between TC's. They have a positive effect on network traffic as a whole. However, the advantages gained from BAM's generally don't favor targeted applications in the network. Simply having dynamic BAM's like G-BAM [27] isn't enough to increase the QoS control that the PSIoT-SDN has on its traffic because it still has to compete for the extra bandwidth.
Integration with a custom SDN BAM solution allows the PSIoT-SDN to passively signal to the SDN Controller which traffic groupings that will take precedence in bandwidth sharing, giving the PSIoT-SDN a more exclusive access to the currently unused bandwidth.
With the simulation we can observe how the SDN controller allocates bandwidth to PSIoT-SDN traffic groups at the network and how the framework responds to variations in bandwidth demand. In order to measure and compare the effects of the SDN integration, the proof of concept will run the PSIoT-SDN, that integrates SDN/ OpenFlow, and the PSIoT-Orch that does not make use of SDN. Figure 9 shows the link load at each simulation event. By redistributing the traffic from aggregators, according to IoT QoS levels, into different traffic classes there is a better use of the overall network bandwidth and also provides the IoT traffic with network-level QoS.
Comparing the effects of the PSIoT-SDN integration, Figure 10 shows the link load of the simulation being executed with and without network-level QoS integration. The integration with the SDN network can utilize up to 100% of the link's capacity and an BAM's are not a novel concept, and different flavours allow for similar bandwidth sharing. However the main advantage of this integration is the ability to share bandwidth according to IoT specific QoS levels, enhancing the level of guarantees in QoS capabilities for IoT traffic. These results display the flexibility of the PSIoT-SDN integrated into the network's management, especially considering how SDN networks have facilitated integration with Application Domain services and more easily allow for third party integration into network management.
The orchestrator implements IoT data QoS by effectively managing transmission rates. The lower the rate of data transmission, the longer the sending of information is delayed. Considering that transmission rates are the determining factor for the QoS enforced on data going trough the PSIoT-SDN framework, an increase in overall network utilization will decrease the overall data transmission delay, providing better QoS.
A relevant consideration concerning the proof of concept topology is that the evaluation considered a single network link. In effect, our objective is to demonstrate that the cumulative edge-QoS and network-QoS control will enhance the overall quality of service for massive IoT traffic. It is also worth to emphasize that BAMs manage bandwidth on a per-link basis [28]. As such, the overall QoS behavior for a path between IoT data producer and consumer will benefit from any link bandwidth improvement over the path. This further validates the proof of concept presented.

FINAL CONSIDERATIONS
The internet has changed how people, machines and physical objects, "things", communicate. The use of "things", devices that communicate with each other and with people has made an impact on the current network technology, creating issues with scalability, security and performance. The solutions to internet quality problems already exist for quite some time. Traffic from specific technologies that have a very large impact on network congestion, like the IoT, can leverage these solutions of routing, geography and bandwidth sharing to better serve it's QoS requirements by creating customized frameworks and protocols.
Factors such as how data is transferred, discovered, shared and consumed are fundamental in creating technology and managing network traffic that better fits the needs of the IoT.
We presented an overview of the requirements and characteristics of IoT data, how devices communicate and how IoT data is transmitted. Based on these attributes, we introduced a framework for IoT traffic orchestration, enabling dynamic QoS capabilities and further integration into the backbone network management, namely SDN.
The framework Orchestrator provides Fog-like IoT data aggregators on the network edge with a familiar Pub/Sub interface for IoT consumers. The IoT data's transmission is managed by IoT specific QoS classes, enforced by data transmission scheduling.
The framework Orchestrator also orchestrates subscribed traffic from all registered IoT aggregators, according to available information on bandwidth and topology and subscription QoS requirements. The QoS capabilities can further be enhanced by integrating into the SDN management.
Resource management and traffic scheduling are often, and in the scenario explored in this paper, a zero-sum game. The framework Orchestrator manages traffic with the effect of providing QoS to high priority IoT data, as asserted by IoT topic subscriptions. This effect is increased when SDN integration allows network bandwidth sharing to be passively managed by the PSIoT-SDN framework, enabling better utilization of the network and enhanced QoS fulfillment for IoT traffic.
The simulation results showed how the shared bandwidth was distributed among the framework's IoT QoS levels, generating a larger throughput of IoT traffic when compared to the same scenario when SDN integration was excluded. With these positive results, the PSIoT-SDN framework is validated and displays its usefulness in managing QoS for IoT traffic.
In summary, the main contribution of the PSIoT-SDN framework is to couple a QoS approach on network edge with a network bandwidth sharing strategy that is SDN-controlled.
The PSIoT-SDN framework stretches across the network, and connects several different IoT networks and domains. Future work can consider expanding on the SDN integration in the Network Domain, tackling the discoverability of IoT topics in the Pub/Sub aggregator nodes, as well as aggregation and data policies in the M2M domain.
The framework was built to integrate into existing internet technologies, working over public networks or with enhanced capabilities in SDN networks. This is an important characteristic to maintain in an IoT framework, considering the fast changing scenario and the heterogeneity of data, traffic and applications.