A Qualitative and Quantitative Evaluation of Differential Signal Strength Fingerprinting Methods

Indoor location systems that rely on WiFi signal strength values from the existing building infrastructure deliver adequate accuracy with no installation cost at the expense of time and effort to populate the signal database. To this end, crowdsourcing has been widely explored to leverage on large volumes of user-collected data; however, mobile devices that report signal strength differently, known as device diversity, limit its applicability in practice. We consider crowdsourced location systems and present a qualitative (in terms of analytical results) and a quantitative (with respect to real-life experimental data) evaluation of several approaches that are robust to diverse devices while focusing on differential signal strength methods.


I. INTRODUCTION
Location awareness has become an integral part of the mobile user's daily life and the wide availability of location information has triggered the advent of innovative locationbased services, while according to Market Research Future the global market is estimated to grow up to USD 80 Billion by 2023 [1]. These services will be increasingly targeting indoor application scenarios as people already spend 80-90% of their time in indoor environments.
Among the wide range of indoor location technologies, WiFi signal strength fingerprinting that may be augmented with other sensor readings is gaining traction. This solution relies on the collection of location-tagged Received Signal Strength (RSS) measurements from the surrounding WiFi Access Points (AP) inside the building, referred to as fingerprints, which are stored in a signal database known as radiomap. Even though this approach has demonstrated comparable or even better accuracy than expensive solutions that require additional hardware, it has not been widely adopted. This is due to the data collection process, which can be tedious and time consuming especially for multi-floor buildings.
To reduce the data collection effort, crowdsourcing has emerged as a viable alternative that leverages user-collected WiFi RSS readings. Even though several crowdsourcing systems have been presented in the literature (see, for example, [2] and references therein), the key challenge that hinders their wider adoption is device diversity [3]. That is, mobile devices report largely different RSS values from a WiFi This work has been supported by the European Union's Horizon 2020 research and innovation programme under grant agreement No 739551 (KIOS CoE) and from the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development.
AP, even if they reside at the same location, due to multivendor WiFi adapters, varying antenna gain, different circuitry packaging, etc. This fact makes the RSS values collected with diverse devices incompatible with each other, thus impeding the combination of all data in a single radiomap.
Some approaches apply data transformations to remove the device dependency from RSS readings, which enables a user to start localizing accurately as soon as he/she enters the building carrying a new device. For instance, the FreeLoc system [4] uses relative RSS values in a Rank Based Fingerprinting (RBF) approach, which ranks the WiFi APs from stronger to weaker RSS. However, the exact RSS values are lost, which hinders fine-grain localization [3]. The Hyperbolic Location Fingerprinting (HLF) approach uses normalized logarithm RSS ratios for all AP pairs in the observed fingerprint [5].
Another data transformation approach uses RSS differences, rather than absolute RSS values, to homogenize the RSS data from multiple devices. For instance, the DIFF method takes the difference between all AP pairs [6]. The Signal Strength Differences (SSD) method subtracts the RSS value of an anchor AP from the other RSS values [7]; however, it is reported that DIFF performs better than SSD for crowdsourced indoor localization [8]. Recently, the Mean Differential Fingerprint (MDF) method proposed to subtract the mean RSS value across all APs in the original fingerprint from each RSS value [9]. It was shown theoretically that MDF delivers exactly the same location estimates as DIFF, but at significantly less computational expense. These differential methods demonstrated promising location accuracy in simulation and small-scale experimental setups; however, still their behaviour has not been fully analyzed and they have not been compared against competing solutions in real-life conditions.
To this end, the contribution of this work is twofold. First, we attempt to gain theoretical insights on differential fingerprinting for crowdsourced indoor localization. We derive analytical models for their respective probability of correct location estimation and employ these models to assess and analyze their performance theoretically for the first time. Second, we conduct extensive experiments using real-life WiFi RSS data from two publicly available datasets and compare differential methods against other approaches. The results validate our analytical models and offer valuable guidelines for the design of crowdsourced localization systems.
The rest of the paper is structured as follows. Section II describes the problem of crowdsourced RSS fingerprinting and formulates existing differential fingerprinting methods. In Section III, analytical expressions are derived for these methods followed by an analysis of their localization performance with respect to various system parameters. Experimental results pertaining to real-life WiFi RSS datasets are discussed in Section IV. Finally, Section V provides concluding remarks.

II. CROWDSOURCED FINGERPRINTING A. Problem Formulation
Traditional fingerprint-based systems operate in two stages, namely the offline phase where the WiFi RSS values are used to create the location-tagged radiomap of the building and the online phase where the radiomap is exploited to compute user location. In crowdsourced systems, data come from multiple heterogeneous devices that report RSS differently, while the user-carried device during localization most likely differs from the crowdsourcing devices.
In this context, the problem of crowdsourced fingerprint localization is formulated next. In the offline phase, a grid of reference locations {L : ℓ i = (x i , y i ), i = 1, . . . , l} are used to discretize the localization area.
in ] T associated with location ℓ i is a vector of RSS readings and r (m) ij denotes the RSS reading from the j-th AP recorded by device D (m) . The fingerprints collected by D (m) form the device-specific radiomap R (m) ∈ Z − l (m) ×n that may partially cover the area, while all devices contribute their respective radiomaps. The naive approach for building the crowdsourced RSS radiomap R ∈ Z − l×n , using all available radiomaps, is by aggregating the RSS values for each AP across all contributing devices M i at location ℓ i , where 1 ≤ M i ≤ M , according to In the online phase, given a new fingerprint s = [s 1 , . . . , s n ] T measured at the unknown location ℓ by the user-carried device D (m ′ ) , the crowdsourced radiomap R is employed to estimate ℓ. For simplicity we use the Nearest Neighbor (NN) approach that determines location as where d 2 i is the squared Euclidean distance between the reference fingerprints r i and the observed fingerprint s.

B. Crowdsourcing with Differential Fingerprints
We assume that the RSS values (in dBm) are given by the simple log-distance radio propagation model where d denotes the physical distance between a WiFi AP and the device, while A provides the RSS value at distance d = 1 m and encapsulates device specific characteristics, such as the antenna gain and transmitter power. The path loss exponent γ depends on the propagation conditions and X ∼ N (0, σ 2 ) represents Gaussian noise that disturbs the RSS values. Using RSS differences removes the constant term A in (3), which makes the differential fingerprints from diverse devices compatible with each other and enables the creation of the crowdsourced radiomap [9]. In the following, we outline the main differential fingerprint approaches.
1) DIFF Approach: Creates the differential fingerprints by taking the difference between all pairwise AP combinations [6]. Therefore, for a single device the DIFF reference fingerprintr i at location ℓ i and the DIFF fingerprint to be localized s are defined as wherer ijk = r ij − r ik ands jk = s j − s k , 1 ≤ j < k ≤ n denote the RSS difference between the j-th and k-th APs in the reference and localization fingerprint, respectively.
Assuming M i devices for crowdsourcing data at location ℓ i , the differential radiomapR ∈ Z l× n(n−1) 2 contains the reference fingerprintr i wherer ijk is computed across all devices using (1) withr ijk . Note that this approach increases the dimension of the original RSS fingerprints as the new fingerprints contain n 2 = n(n−1) 2 RSS differences. Localization with the NN method is performed by replacing d 2 i in (2) with 2) SSD Approach: Creates the differential fingerprints by subtracting the RSS value of an anchor AP from the other RSS values in the original fingerprint [7]. Without loss of generality, we assume that ρ is the anchor AP and we define for a single device the SSD reference fingerprintř i at location ℓ i and the SSD fingerprint to be localizedš aš . . , n, j = ρ denote the RSS difference between the j-th AP and the anchor AP ρ in the reference and localization fingerprint, respectively.
Assuming M i devices for crowdsourcing data at location ℓ i , the differential radiomapŘ ∈ Z l×(n−1) contains the reference fingerprintř i whereř ij is computed across all devices using (1) withř (m) ij . Note that in this case the dimension of the new fingerprints is decreased as they contain the n − 1 RSS differences that are independent. Localization with NN is performed by replacing d 2 i witȟ 3) MDF Approach: Creates the differential fingerprints by subtracting the mean RSS value pertaining to all APs from each RSS value in the original fingerprint [9], [10]. Thus, for a single device the MDF reference fingerprintr i at location ℓ i and the MDF fingerprint to be localizeds are defined as wherer ij = r ij −r i ands j = s j −s, j = 1, . . . , n denote the RSS difference between the j-th AP and the mean RSS value in the reference and localization fingerprint, respectively. The mean RSS values in fingerprints r i and s are given bȳ r i = 1 n n j=1 r ij ands = 1 n n j=1 s j , respectively. Assuming M i devices for crowdsourcing data at location ℓ i , the differential radiomapR ∈ Z l×n contains the reference fingerprintr i wherer ij is computed across all devices using (1) withr (m) ij . Note that the MDF fingerprints have the same dimension as the original RSS fingerprints. Localization with NN is performed by replacing d 2 i with

III. ANALYSIS OF DIFFERENTIAL FINGERPRINTS
To gain insight in the behaviour of differential fingerprints, we derive analytically the probability of correct location estimation and then analyze their performance for crowdsourcing.

A. Probability of Correct Location Estimation
Let ℓ 1 and ℓ 2 be two neighbouring locations in the localization area. We consider two devices, i.e., D (1) and D (2) , that collect RSS values at both ℓ 1 and ℓ 2 and we assume that r 1j and r 2j computed with (1) are deterministic because each has been averaged over a sufficiently large number of samples collected with both devices to filter out the noise. Therefore, the corresponding RSS fingerprints r 1 and r 2 in the crowdsourced radiomap are constant vectors. Moreover, for simplicity, we assume that during localization the user is located at ℓ 1 carrying device D (1) and observes the RSS fingerprint s (1) . The fingerprint s (1) is a normally distributed random vector, i.e. s (1) where Σ = σ 2 I n is the covariance matrix, while σ 2 is the variance of the Gaussian noise that disturbs the RSS values and I n is the identity matrix.
The objective is to analytically derive the probability that the NN localization method will return the correct location ℓ 1 , instead of the incorrect location ℓ 2 when RSS, DIFF, SSD, or MDF fingerprints are used, respectively. Next, we generalize our results for areas where the radiomap contains several location fingerprints.

1) Analytical Model for RSS Fingerprints:
In the case of traditional RSS fingerprints, the NN method will return the correct location ℓ 1 only if the condition d 2 1 ≤ d 2 2 is satisfied. In other words the unknown user location is correctly identified only if the distance between the observed fingerprint s and the correct location fingerprint r 1 is smaller than the distance between the observed fingerprint and the incorrect neighbouring location fingerprint r 2 .
We follow an approach similar to [11] and starting from the above condition we can easily show that 1 where β j = (r 2j − r 1j ) and γ j = (r 2 1j − r 2 2j ). For convenience, we use vector notation to rewrite (10) as where β = 2[β 1 , . . . , β n ] and γ = n j=1 γ j . The random variable C = βs + γ is normally distributed, as a linear function of the multivariate normal vector s, i.e.
Thus, the probability of correct location estimation, when the NN method compares the observed fingerprint with just two location fingerprints in the radiomap, is given by In a real system where the radiomap is expected to contain several location fingerprints, depending on the size of the area and the density of the reference locations, the probability of correct location estimation P C can be calculated as where the variable C i corresponds to the condition d 2 1 ≤ d 2 i , i = 2, . . . , l. Although this analytical model is based on the assumption that the random variables C i are independent, which is not true, it still provides a good approximation of the probability of correct location estimation [11].
2) Analytical Model for MDF and DIFF Fingerprints: In our previous work, it was shown that the distancesd 2 i between the MDF fingerprintsr i ands are proportional to the distances d 2 i between the DIFF fingerprintsr i ands, i.e.,d 2 . Thus, the ordering of the candidate locations ℓ i ∈ L is not affected when either MDF or DIFF fingerprints are considered. Consequently, the user locations estimated with NN are exactly the same and we derive the analytical model only for MDF.
Starting from the conditiond 2 1 ≤d 2 2 and using Theorem 1 in [9], which provides the relation of the distanced 2 i between MDF fingerprints with respect to the distance d 2 i between the corresponding RSS fingerprints, it can be shown that where β j = (r 2j − r 1j ) and δ j = (r 2 1j −r 2 1 − r 2 2j +r 2 2 ). Using vector notation, (15) is equivalent to where β = 2[β 1 , . . . , β n ], J = I − 1 n e T e, e = [1, . . . , 1] and δ = n j=1 δ j . The random variable Q = βJs + δ is normally distributed 2 , i.e. Q ∼ N (µ Q , σ 2 Q ) with mean and variance (µ Q , σ 2 Q ) = (βJr The probability of correct location estimation, using the NN localization method with MDF fingerprints, in the two-location case and the multiple location setup respectively, are given by (18) 3) Analytical Model for SSD Fingerprints: Starting from the conditionď 2 1 ≤ď 2 2 and using Theorem 2 in [10], which provides the relation of the distanceď 2 i between SSD fingerprints with respect to the distance d 2 i between the corresponding RSS fingerprints, it can be shown thať where β j = (r 2j − r 1j ), β ρ = (r 2ρ − r 1ρ ) and ǫ j = (r 2 1j + r 2 1ρ − r 2 2j − r 2 2ρ − 2r 1ρr1 + 2r 2ρr2 ), with ρ being the anchor AP. Again, using vector notation we may rewrite (19) as Finally, the probabilities that the NN localization method returns the correct location when it compares the observed SSD fingerprint with just two SSD location fingerprints and several SSD location fingerprints respectively are given by

B. Performance Evaluation
We adopt the simple localization setup depicted in Fig. 1a  [8], [11]. There are l = 9 reference locations (marked with circles) that are uniformly spread over a square grid. The WiFi APs (marked with triangles) are deployed in the perimeter and we start out with n = 4 APs that are placed at the four corners of the area, while the maximum number is n = 16 APs.
We assume that D (1) is the first crowdsourcing device and the RSS values r (1) ij at the reference locations are given by the propagation model of (3). We use typical values A = −22.7 dBm and γ = 3.3 for the model parameters. We also assume that r (1) ij is deterministic because it has been averaged over a sufficiently large number of samples to filter out the noise. These RSS values, generated at all 9 locations, constitute the device-specific radiomap R (1) . In addition, a number of crowdsourcing devices D (m) , m = 2, . . . , M also cover the whole area and we assume a linear relation between the RSS values reported by any device D (m) and the corresponding RSS values of device D (1) such that where (α 1m , β 1m ) are the linear fitting parameters between devices D (m) and D (1) [8]. All M devices contribute their respective radiomaps R (m) and the crowdsourced RSS radiomap R is created by aggregating the RSS values according to (1). We now employ the analytical models in (14), (18), and (22) for RSS, MDF, and SSD to analyse the performance of crowdsourced localization using differential fingerprints. We start out with two devices for crowdsourcing, i.e., D (1) and D (2) with (α 12 , β 12 ) = (0.95, 10), and we assume that a user carries device D (1) during localization while he/she resides at the central location ℓ 1 = (2, 2). We generate the RSS fingerprint s by taking the RSS location fingerprint r (1) i and disturbing it with Gaussian noise. Then, we compute the probability of returning the correct location ℓ 1 .
First, we investigate the effect of varying number of APs. In Fig. 1b we observe that the performance of the traditional RSS fingerprints is poor when few APs are considered; however, for more APs (i.e., n > 8) they perform better or equally well to SSD and MDF, respectively. Moreover, peaks occur for certain subsets of APs, i.e., n ∈ {4, 8, 12, 16}. In these cases the uniform distribution of APs creates symmetries that mask the effect of device diversity 3 . Similar behaviour was reported with simulations in [8]. Nevertheless, in large-scale real-life setups, the APs are neither expected to be densely deployed nor evenly distributed. In contrast, device diversity does not have any impact on MDF and its performance improves. SSD is inferior due to the lower dimension of the SSD fingerprints, while the error bars indicate the standard deviation around the mean value when using different APs as anchors. Fig. 1c plots the probability of correct location estimation for increasing noise standard deviation σ with n = 6 APs. It is evident that even under low noise conditions RSS fingerprints perform poorly. In contrast, the differential approaches are more robust to noise and their performance degrades smoothly. Although MDF and SSD achieve similar performance when σ = 1 dBm, beyond that point the MDF approach outperforms SSD by around 10%-20% when σ ≤ 6 dBm.
The effect of growing number of crowdsourcing devices is depicted in Fig. 1d. For M = 1, only device D (1) is used for building the radiomap, while the same device is used for localization. Therefore, it is not surprising that RSS performs best. However, when more devices are considered, the performance of RSS drops rapidly and then starts to gradually improve when M ≥ 6. Still the performance of RSS is well below the differential methods. Both MDF and SSD have a slight linear degradation as more devices contribute to crowdsourcing, while MDF outperforms SSD by around 20%.  [12]. We report the results for those floors in various buildings where around 100 or more test fingerprints are available.

B. Evaluation Process and Performance Metric
We use both real-life datasets to compare the traditional RSS fingerprints against the differential fingerprinting methods SSD [7] and MDF [9], as well as other approaches such as HLF [5] and RBF that is similar to the rank-based method used in FreeLoc [4]. For SSD, the AP that exhibits the least average deviation of RSS values over the whole localization area is set as anchor. We followed the same approach for the HLF method to avoid the computational overhead of the original method that uses all pair-wise AP combinations (e.g., in the UJIIndoorLoc dataset the original HLF fingerprints would contain more than 3.570 values, thus incurring 40x more computations in NN localization). The DIFF method [6] is omitted as it provides exactly the same accuracy with MDF at a considerably higher computational cost.
We compute the localization error defined as the Euclidean distance between the actual user location and his/her location estimated with the NN approach. We report statistics for the cumulative error distribution and the central mark in the box plots indicates the median error, the box edges correspond to the 25th and 75th percentiles, and the whiskers extend to the 5th and 95th percentiles, respectively. The bar charts present the mean localization error together with the 95% confidence interval given by ±1.96σ e / |T |, where σ e is the standard deviation of the error and |T | is the number of test samples. The 95% confidence interval indicates that the mean error falls within the interval with a high degree of certainty.
The results for the KIOS dataset are illustrated in Fig. 2. First, we focus on one device for localization (i.e., HTC Desire smartphone) and consider all five devices for crowdsourcing in Fig. 2a. We observe that the MDF and SSD approaches outperform the HLF and RBF methods. MDF is slightly better than SSD with the median error being well below 2 m. Even though the performance of RSS fingerprints is improved with more crowdsourcing devices, still they fail to deliver the same level of accuracy as MDF. This observation is confirmed in Fig. 2b where we study the error of localizing the Samsung Nexus S device for increasing number of crowdsourcing devices M . When M = 1, only the Nexus S is used to build the radiomap and the RSS fingerprints achieve the lowest localization error. However, in case M = 2 their performance degrades significantly, as the second device reports RSS very differently from the Nexus S. This effect is gradually smoothed out as more devices contribute data. The behavior of RSS is in agreement with our analytical findings in Fig. 1d. Figure 3 depicts the localization error using data from Building 'x' and Floor 'y', denoted BxFy for brevity, of the UJIIndoorLoc dataset. The MDF method outperforms significantly all other methods at B1F1 as shown in Fig. 3a. This is explained by the number of crowdsourcing devices (i.e., 5 devices) and the difficulty in selecting the anchor AP in such large setups for the SSD and HLF methods. Regarding B0F1, where two devices are used for crowdsourcing, the MDF method still provides lower error than SSD and HLF by 1 m and RBF by 2.5 m, as shown in Table I. The RSS method performs equally well with MDF because the devices in both cases are Android devices that seem to report RSS similarly. Figure 3b plots the localization error at B1F2 for The results are averaged for ten scenarios with randomly selected subsets of APs. The RSS fingerprints do not have the oscillations predicted analytically (see Fig. 1b) due to the irregular deployment of the WiFi APs. Even though the performance of RSS may seem to be close to MDF, the median error in Table I suggests that in some cases, e.g., B1F1 or B1F2, the RSS fingerprints may lead to lower accuracy.

V. CONCLUSIONS
We investigate the effectiveness of WiFi RSS data transformation methods against device diversity for crowdsourced localization systems. Our key findings are i) RSS values should be avoided as in most cases they degrade the quality of the crowdsourced radiomap and lead to higher errors; ii) the use of RSS values could be justified in areas covered by a large number of WiFi APs, when the volume of crowdsourcing devices is high, or if the devices are not highly heterogeneous, e.g., Android devices equipped with similar WiFi adapters; iii) differential methods deliver higher accuracy compared to other transformation methods; and iv) the MDF method achieves the best performance under all crowdsourcing scenarios. These findings can be viewed as design guidelines for real systems depending on the size of the area and number of WiFi APs, the volume and the heterogeneity of the crowdsourcing devices.
Compared to traditional RSS, the MDF method addresses device diversity with a marginal overhead (i.e., to compute the mean and subtract from the RSS values in the original fingerprint), while preserving the fingerprint dimension. Importantly, MDF can be applied as a data pre-processing technique regardless of the localization algorithm, e.g., NN or more advanced fingerprint matching approaches. As part of our future work, we plan to integrate the MDF method into our Anyplace 4 crowdsourced service [13] to further assess its value and measure the power consumption on real mobile devices.