Fog Computing: A Linguistic Approach for Knowledge Inference in Wearable Devices

. Fog Computing has emerged as a new paradigm where the processing of data and collaborative services are embedded within smart objects, which cooperate between them to reach common goals. In this work, a rule-based Inference Engine based on fuzzy linguistic approach is integrated in the smart devices. The linguistic representation of local and remote sensors is deﬁned by protoforms, which conﬁgure the antecedents of the rules in the Inference Engine. A case study where two inhabitants with a wearable device conduct activities in a Smart Lab is presented. Each wearable device infers the daily activities within the wearable devices by means of the rule-based Inference Engine.


Introduction
Since last years, Ambient Intelligence (AmI) [1] and the Ubiquitous Computing (UC) [2] have developed new computing paradigms integrating intelligence systems in the sensor and environments for an ambient assisted living [3]. These trends have enabled analyzing daily human activities from computer sciences by means of pervasive and mobile computing [4]. In this way, activity recognition has resulted a challenging research topic because of supervising elderly people to stay with the best quality of life as long as possible in their sustainable, healthy and manufacturing homes [5], whose percentage of population over 65 up to 15% [6].
Currently, recent paradigms, such as Edge Computing [7] or Fog Computing [8] edges the data and services within the devices where data are collected providing virtualized resources and engaged location-based services to the edge of the mobile networks [9]. This new perspective translates the focus on Internet of Things (IoT) [10], from Cloud Computing with centralized processing [11], to collaborative networks where the smart objects interact with each other and cooperate with their neighbors to reach common goals [12,13]. In concrete, Fog Computing has involved a great impact between ambient devices [14] and wearable devices [15].
In order to integrate intelligence systems in smart objects and smart environments by means of Fog Computing, several key aspects have to be handled: -Real time ubiquitous computing. The adequate distribution of services between smart objects is crucial to provide sensitivity to real time [16] where the most important part of information processing is distributed in different central processing units [17]. -Knowledge representation of data streams from heterogeneous sensors. A wide range of heterogeneous sensors are deployed in smart objects being necessary: (i) standard structural models, such as SensorML [18], developed by Open Geospatial Consortium, (ii) semantic model ontologies, such as the W3C Semantic Sensor Networks (SSN) specifications [19] or (iii) ad hoc proposals which provide scientist interoperability [20]. -Collaborative knowledge generation. Fog Computing requires by information processing of sensors to generate richer and higher-level information [21] in the devices where data are collected. This issue requires that raw data from sensors have to be analyzed, summarized and merged with other sensor information by means of context-aware computing [22].
Based on these issues, in this work, we describe a straightforward approach for the fusion and reasoning of knowledge within smart objects using a formal linguistic representation from the stream of data generated within the environment. In order to illustrate the usefulness and effectiveness of our proposal for integrating a fuzzy linguistic representation in Fog Computing, we present a case study where two inhabitants with a wearable device conduct activities in a Smart Lab and the inference of daily activities is computed within the wearable devices using a linguistic representation of local sensors.
The remainder of the paper is structured as follows: in Sect. 2, the proposed approach for distributing and inferring fuzzy linguistic terms in smart objects is presented; in Sect. 3, a case study of daily living activities in an intelligent environment is presented and the approach for ambient and wearable sensors is evaluated. Finally, in Sect. 4, conclusions and future works are pointed out.

Background
In this work we introduce an intuitive linguistic representation of data streams by means of protoforms based on an extension of temporal linguistic terms [23,24]. Protoforms were proposed by Zadeh [25] as useful knowledge model for reasoning [26], summarization [27] and aggregation [28] of data under uncertainty. The managing of uncertainty and vagueness in intelligent environments has been described as a key issue to obtain an high performance and results [29], in particular for Activity Recognition [30].
On the other hand, in Fog Computing, the Real-Time distribution of collaborative information and knowledge is required [31] to provide a scalable approach in which the heterogeneous sensors are distributed to dynamic subscribers in real-time. In this contribution the smart objects are defined as both sources and targets of information using a Publish-Subscribe model [32]. In concrete, the smart objects interact with each others describing their sensor streams by means of protoforms.
Finally, a rule-based Inference Engine based on the fuzzy linguistic approach [33] is integrated in the smart objects, such as wearable devices, to generate higher-level information. The antecedents of the rules are defined by protoforms under a linguistic representation of local and remote data streams. This inference model is close related to the Mamdani inference method [34], although the degree of terms are replaced by the degree of protoforms. The rules can be defined and configured by means of remote services enabling devices being remotely programmed.

Approach for Generating and Distributing Protoforms in Real-Time Collaborative Environments
The main contribution of this paper is to provide a straightforward approach for generating and distributing protoforms in real-time collaborative environments. The approach is based on the definition of intuitive protoforms for representing sensor streams and the integration of a fuzzy Inference Engine within smart objects.
In Fig. 1, the generation of protoforms, distribution under a Real-Time Publish-Subscribe model and the inference engine using a fuzzy linguistic approach are shown. The components are further detailed in next sections.

Protoforms of Sensor Streams
In this section, we resume a previous work [23], where a linguistic summary of sensor streams were proposed using protoforms. Based on our linguistic approach, each sensor data stream s j is composed of a set of measures represents the raw value that measures the sensor. For each sensor data stream s j , we associate a fuzzy linguistic terms V r to describe the measures of the sensor. Second, we associate fuzzy linguistic terms T k with the temporal component of the data stream, defined by the distance Δt j i = t * − t j i , t * > t j i from a reference point of time t * to the timestamp of the measurement t i . Third, we aggregate both terms computing the relevance of a linguistic value term V k (v) in a fuzzy temporal term T k (t) using the t-norm and co-norm operators: The linguistic terms T k and V k which are instanced in the sensor stream s j represent a protoform in the shape of: s j is T k V r . For example, the protoform instance: Motion is now low.

Fuzzy Inference Engine with Distributed Protoforms Within Smart Objects
In this section, we describe a Fuzzy Inference Engine with Distributed Protoforms (FIEDP) within Smart Objects. The fuzzy based-rule systems are based on the fuzzy set theory, which traditionally represents the knowledge of human experts by means of fuzzy IF-THEN rules.
In this work, we include the protoforms, described previously in Sect. 2.1, as antecedents and consequent of the IF-THEN rules. The aim is inferring knowledge from the sensor in an interpretable way without handling with raw data streams. In this way, the linguistic summary of sensor provided by protoforms enables describing fuzzy rules and inferring knowledge with better human understanding.
Under our approach based on Fog Computing, each smart object integrates a Fuzzy Rule-Based Inference Engine, where the instances of the Data Base (DB) and the Knowledge Base (KB) are defined by protoforms. The are shown in Fig. 1.
Synchronized Data Base of Protoforms. The Data Base (DB) of each smart object is composed by local and remote protoforms. The local protoforms are calculated from local sensor streams, where the degree is calculated by the approach described in Sect. 2.1, whilst the remote protoforms are calculated as consequents in remote smart objects regarding next Sect. 2.2.
A Protoform Service deployed in each smart object defines the instances of the local protoforms. The local protoforms enable us configuring different linguistic summaries for each local sensor stream. These protoforms can be subscribed by other smart objects, in order to obtain a linguistic representation of a remote sensor. So, every time that a given data stream updates a new measure, the Fuzzy Rule Based Inference Engine: (i) updates the Data Base calculating the degree of the local protoform, and (ii) sends an event with new degree to the remote subscribers.
Knowledge Base of Protoform Rules. The Knowledge Base (KB) for each smart object is composed of rules of protoforms. The antecedents A ij of each rule R i correspond on local or remote protoforms integrated in the Data Base.
where A i is a protoform related to the linguistic terms of a sensor in the shape of s j is T k V r , based on the description of Sect. 2.1.
The consequents C i is defined as a new inferred protoform, whose degree is calculated by concatenating the antecedents A i1 . . . andA in using fuzzy conjunctive operators (t-norm) ∩ and aggregating the same consequents using fuzzy disjunctive operators (t-conorm) ∪: A Rule Subscription Service deployed in each smart object allows adding or removing instances of rules. Each rule enables us defining new shaped-protoform consequents and distributing the inferred degrees to the subscriber in real time. In this way, the subscribers can keep a real-time synchronization of remote protoforms collected in their Data Bases.
Finally, the Fuzzy Rule Based Inference Engine updates the degrees of inferred protoform consequents for each rule when an antecedent in the Data Base is triggered. This occurs when: (i) the local data stream updates a new measure for local protoforms, or (ii) a degree of a shaped-protoform consequent from a remote subscription is received.

Case Study
In this section, a description of a scene of daily activities is carried out by two inhabitants in a Smart Lab. Inhabitants wear a wearable device and the Smart Lab integrate several ambient sensor in several smart objects. The description of inhabitant and ambient sensors is modeled under the fuzzy linguistic approach using protoforms. Moreover, the wearable devices integrate the proposed Fuzzy Inference Engine receiving remote protoforms, which are connected under a subscription model to ambient objects. The aim of this case study is evaluating the capabilities of wearable devices for inferring the daily activity of each inhabitant in real time.
The case study was carried out in the smart lab of the ACETIC (Center for Advanced Studies in Information Technology and Communication) of University of Jaen 1 .
At the scene: (1) Inhabitant A in the bed, wakes up and brushes his teeth.
(2) Inhabitant A goes to the kitchen, prepares breakfast with the kettle and cereal and sits in the kitchen to have breakfast. The video of the scene is located in the next URL 2 .

Sensors
In order to describe the daily activity carried out by the two inhabitants, we have integrated several ambient sensors to convert traditional objects into smart object: -Motion Cookies. They are movement sensors located in objects, which notify of a specific patter movement, such as brushing teeth or diking a water, to a base station called Mother Sense by means of Z-Wave protocol. Receiving the pattern events is enabled by means of a REST-oriented API. Three cookies have been installed within two cups and a toothbrush respectively. -Pressure sensor. A pressure sensor is installed at the sofa and bed, which notifies to a base station by means of Z-Wave protocol. -Bluetooth Beacons. Six Bluetooth Beacons, in concrete Estimote sensors, are located in the sofa, bathroom, bed and glasses respectively. They enable wearable devices with Low Bluetooth Energy calculating the distance and proximity from inhabitants to the rooms and objects.
The wearable devices of an inhabitants are smart watches with Android Wear, specifically the model LG Urban Watch. In this case scene, we have integrated two sensors from the smart watches: -Accelerometer sensor. It measures the linear acceleration in m/s 2 representing the inhabitant motion. -Bluetooth Low Energy (BLE). It enables smart watches obtain the proximity in dBm based on the RSSI signal to a Bluetooth Beacon, as well as the estimated distance in meters.

Protoforms for Sensor and Rule Definition
At the case scene, we have defined several protoforms to describe the sensor streams from ambient objects and smart watches using a linguistic approach. First, to instance the protoforms, an expert defines the variables and linguistic terms related to each sensor stream, which is shown in Table 1. On the first hand, to translate the data from sensor of accelerometer and BLE from wearable devices to linguistic terms, we have defined their membership functions using trapezoidal function. It is a common representation to define a membership function in continuous domains, which is achieved by a 4- tuple TS(a,b,c,d) where b and c indicate the point in which the membership value is 1, with a and d indicating the left and right limits of the definition domain of the membership function. On the second hand, the ambient devices (mother and zwave base) translate binary data from crisp sensor (cookies and pressure of sofa and bed) to protoforms using a easygoing transformation {on, of f } → {0, 1}. The fuzzy temporal term for calculating the degree of protoforms regarding with Sect. 2.1 has been modeled by the short-term now with the trapezoidal function μ T k (Δt) = T S(0, 0, 2, 4) defined in seconds. We note that to simplify, we have missed now in the description of all protoforms.
Second, the protoforms are distributed under subscription model under a publish model. The distribution of changes of the degree have been implemented with an Object Oriented Middleware, ZeroC Ice [32], which provides a compressive framework for using subscriptions and sending data through channels of real-time events. Based on the principles of Fog Computing related on this work, the events from ambient devices and objects have been spread to smart watches in real time. So, in the scene, the events from cookies and binary sensors are received in their base stations, which have been integrated in the middleware to spread the events to smart watches using the TCP protocol by WiFi connection. Third, a Fuzzy Inference Engine is deployed in each smart watch. Based on the daily activities carried out by inhabitants at this case scene, the Knowledge Base has been composed by the next rules: (

Results
In this section, we present the time-line related to each inhabitant by means of the smart watch and the inference of activities in the scene. First, in Fig. 2, three time-lines with the degree of protoforms for each ambient and wearable sensor are shown. Although the degrees of local protoforms of inhabitant may seem chaotic, they describe the basic antecedents to infer activities in the Inference Engine in each smart watch.
Second, in Fig. 3, two time-lines with the degree of inferred protoforms for both inhabitants and ground-truth activities are shown. They represent the point of view at the scene of each inhabitant A and B. For example, the fact that the inhabitant B is sleeping is related to the low motion, proximity to bed and pressure sensor of bed. Later, brushing teeth of inhabitant B is inferred by the cookie sensor from toothbrush as well as the strong movement and the proximity to bathroom. Inhabitant A appears in the scene while inhabitant B is having breakfast. Inhabitant A enters into kitchen and takes the cup A for drinking. The proximity related to the cups A and B allows Inference Engine detecting which one is drinking from each cup. When inhabitant A sits down on sofa and drinks, it is inferred by the activation of ambient sensors of the cup and the sofa together with the proximity to the sofa. Finally, when inhabitant B sits down on the sofa too, and at the same time inhabitant A drinks, the inferred actions of drinking is detected by both smart watches due to the proximity to the cup and relevant movement of both, although with a higher degree in the correct inhabitant A.

Limitations of the Work
The main limitation of the work is related to the current weak-spots of smart watches. Keeping alive a permanent WiFi connection for receiving ambient events in real-time is not trivial. They have been programatically strained to not closing the network connectivity, which is the default internal policy of the smart watches for reducing battery consumption. So, in real environments, keeping real-time connectivity could not be sensible for long time.  The second issue is the delay and discontinuity of the proximity of BLE beacons. The need for some seconds to be aware of the proximity with rooms. Moreover, in regard to proximity to objects, we also note that locating beacons in moving objects, such as cups, has provided a better accuracy for detecting hand activities, but accuracy is affected while walking and it is necessary some seconds of motionless to recover the measurement.

Conclusions
In this work, we have described a fuzzy linguistic approach for handling Fog Computing in wearable devices, which provide a new perspective and accuracy in activity recognition.
For that, we have extended the use of protoforms of previous works [23,24], where linguistic expressions for describing data streams were proposed, with a Fuzzy Inference Engine integrated within wearable devices. It has provided a new perspective based on Fog Computing, where the inference of activities is deployed within wearable devices. So, the intelligence of recognizing activities has been focused on the devices of user, keeping the privacy of processing the personal sensor streams away from cloud services.
On the case scene presented in results, we have deployed proximity beacons both in room and smart objects detecting the proximity of inhabitants when interacting with objects. However, improvements in movement and accuracy are needs in next generation of devices to guaranty a higher detection of a closer proximity in real-time. Moreover, real-time distribution of ambient events to smart watches has been developed with WiFi under TCP protocol. In real environments, the connectivity by Bluetooth Low Energy instead of WiFi could provide a better battery life of smart watches. Next generation of middleware for IoT should deal with this problem.
In future works, we will focus on describing a longer dataset of daily activities in smart environments using wearable devices in order to provide a better accuracy in Multioccupant Activity Recognition.