Definition of QoE Fairness in Shared Systems

User-centric service and application management focuses on the quality of experience (QoE) as perceived by the end user. Thereby, the goal is to maximize QoE while ensuring fairness among users, e.g., for resource allocation and scheduling in shared systems. Although the literature suggests to consider consequently QoE fairness, there is currently no accepted definition of QoE fairness. The contribution of this letter is the definition of a generic <italic>QoE fairness index</italic> <inline-formula> <tex-math notation="LaTeX">$F$ </tex-math></inline-formula>, which has desirable key properties as well as the rationale behind it. By using examples and a measurement study involving multiple users downloading web content over a bottleneck link, we differentiate the proposed index from QoS fairness and the widely used Jain’s fairness index. Based on results, we argue that neither QoS fairness nor Jain’s fairness index meet all of the desirable QoE-relevant Properties, which are met by <inline-formula> <tex-math notation="LaTeX">$F$ </tex-math></inline-formula>. Consequently, the proposed index <inline-formula> <tex-math notation="LaTeX">$F$ </tex-math></inline-formula> may be used to compare QoE fairness across systems and applications, thus serving as a benchmark for QoE management mechanisms and system optimization.


Definition of QoE Fairness in Shared Systems
Tobias Hoßfeld, Lea Skorin-Kapov, Poul E. Heegaard, and Martin Varela Abstract-User-centric service and application management focuses on the quality of experience (QoE) as perceived by the end user.Thereby, the goal is to maximize QoE while ensuring fairness among users, e.g., for resource allocation and scheduling in shared systems.Although the literature suggests to consider consequently QoE fairness, there is currently no accepted definition of QoE fairness.The contribution of this letter is the definition of a generic QoE fairness index F, which has desirable key properties as well as the rationale behind it.
By using examples and a measurement study involving multiple users downloading web content over a bottleneck link, we differentiate the proposed index from QoS fairness and the widely used Jain's fairness index.Based on results, we argue that neither QoS fairness nor Jain's fairness index meet all of the desirable QoE-relevant Properties, which are met by F. Consequently, the proposed index F may be used to compare QoE fairness across systems and applications, thus serving as a benchmark for QoE management mechanisms and system optimization.

I. MOTIVATION AND SCOPE
Q UALITY of Experience (QoE) is "the degree of delight or annoyance of the user of an application or service."[1].It is generally accepted that the quality experienced by a user of a networked service is dependent, in a non-trivial and often non-linear way, on the network's QoS.Moreover, the QoE of different services is often different given the same network conditions; i.e., the way in which QoS can be mapped to QoE is service-specific.
Standards such as [2] specify how to compute various QoS metrics and highlight the need to consider customer QoE targets.However, fairness aspects are not considered.From an operator's point of view, QoE is an important aspect in keeping customers satisfied, and thus decreasing churn.This has lead to a number of mechanisms for QoE-driven network resource management, aimed at maintaining quality above a certain threshold for every user (or for example "premium" users).An issue common to all those efforts is that of dividing the available resources among users so as to maintain a satisfied customer base.While a significant amount of literature advocates the need for addressing QoE fairness, what is still missing is a widely accepted metric for this purpose.In this letter we define a number of desirable index properties and introduce a novel QoE fairness index F that meets these properties.While we argue the need to distinguish QoE fairness from QoS fairness, the aim of this letter is not to address underlying influence factors and dimensions of QoE, nor how they are measured.Rather, we focus on formulating a general fairness metric which satisfies QoE-relevant properties, assuming that estimated QoE values are known.

A. Notion of Fairness in Networking
In networking, fairness in resource allocation and scheduling is either linked to sharing resources evenly amongst the entities, or scaling the utility function of an entity in proportion to others.Flow based resource sharing, e.g., max-min fairness, is the fundament of the design of TCP and fair queuing scheduling approaches.The proportional fairness is the ratio between maximum resource consumption and its average resource consumption so far, and weighted proportional fairness is defined [3] as the scaling of an entity's utility function relative to others, such that the entities will allocate flow rates so that the cost they cause will equal the weight they choose.
Any resource scheduling allocation between different entities (users, applications, flows/sessions, bitstreams) must have a notion of fairness.A QoS fairness index should reflect the distance between the actual and the idealised allocation.The fairness should be relative to the resource unit x i which is allocated to entity i relative to the other entities.Various measures have been proposed, both for measuring short-term (temporal) and long-term fairness.The most frequently used QoS fairness metric is Jain's index [4], which approximates the ratio between the squares of the first and second order moments of the resources x i allocated for entity i .Jain's index is primarily for assessing long-term fairness (e.g., averaged per user, session), but can also evaluate short-term fairness by considering the sliding window average of x i .Other indexes which (partly) measure the fairness of shared resources are: variation, coefficient of variation, ratio between the maximum and minimum x i (Max-Min index).

B. Fairness From the User's Perspective
While QoS fairness has been well established in the networking community, much less focus has been put on considering fairness from a user-oriented perspective.Briscoe has argued that equal bitrate, or any flow-rate definition of fairness, is ultimately unfair [5], and that fairness should be considered from the point of view of congestion costs or user benefits.Concerning the latter, while the benefit of a data transfer can be assumed to increase with flow rate, the shape and size of the function relating the two is subjective and application specific.Following this perspective, recent letters have argued that a QoS fair system is not necessarily QoE fair, e.g.[6], given the lack of consideration of underlying QoE models.Such models specify the relationships between user-level QoE and various application-layer performance indicators (e.g., file loading times, video re-buffering) or influence factors such as device capabilities, context of use, network and system requirements, user preferences, etc.
As an example we consider QoE fairness in the context of bottleneck link sharing among adaptive video streams, whereby the on/off nature of flows results in inaccurate clientside bandwidth estimation and leads to a potential unfair resource demand [6], [7].De Cicco et al. [8] propose a clientside algorithm which avoids on/off behavior until reaching the highest possible playback quality.However, while they focus on QoS fairness, the approach still faces such problems as heterogeneous user devices, thus the issue of achieving QoE fairness remains.Mansy et al. [6] argue that typical flow-rate (QoS) fairness ignores user-level fairness and is ultimately unfair, thus proposing a QoE fairness metric in the range [0; 1] based on Jain's fairness index.Further, Petrangeli et al. [7] incorporate the notion of maximizing fairness, expressed as the standard deviation of clients' QoE, into a novel rate adaptation algorithm for adaptive streaming.
Cofano et al. [9] focus on network-assisted strategies for HTTP adaptive streaming, and quantify fairness as follows.SSIM values are computed for observed videos and compared to SSIM values computed for a theoretically optimal solution, with the difference computed as an RMSE value to quantify fairness.While this approach is valid for the given case, it depends heavily on formulation of the optimization problem (which may lead to an unfair system), and cannot be used as a general QoE fairness index.
The aforementioned approaches aim to exploit QoE fairness e.g., for optimized QoE-driven network resource allocation.We focus instead on defining a metric independent of the underlying service and QoE.We define a generic QoE fairness index to serve as a benchmark when comparing different resource management techniques in terms of their fairness across users and services.Our contribution is the definition of such an index which meets a set of desired properties, along with a rationale for it and examples of how it differs from the notion of fairness usually seen in queueing systems.

II. DEFINITION OF THE QoE FAIRNESS INDEX A. Notion
We consider a shared system of users consuming a certain service.For each service there is a set of QoS parameters (of various key QoS influence factors on QoE) given in a vector x.There exists a mapping function Q from the QoS parameters in the set x to a QoE value y, For user i the corresponding QoS parameters are x i , with QoE value y i .The L is the lower bound and H the upper bound of the QoE value, e.g.L = 1 and H = 5 when using a 5-point ACR scale and Q(x) is the mean opinion score (MOS) value for QoS x.The mapping function is required (a) to map QoS parameters into the QoE domain [L; H ] and (b) to uniquely map QoS to QoE; it does not need to be monotonic.In the literature, those QoE models are often derived by subjective user studies, while typically only the MOS scores are used.However, other QoE metrics (like median, quantiles, etc.) may be especially of interest for service providers [10] which may be reflected by Q.Typically, the QoE of the entire session per user is modeled.This session-based QoE allows to derive long-term QoE fairness.There are also attempts to capture instantaneous QoE [11] which allows to derive short-term QoE and consequently short-term QoE fairness.
In a system with several users, the set of observed QoS parameters over all users (in the steady state or the transient phase) is a random variable (RV) X, which means that after the mapping the QoE value Y is also a RV.A fairness index is then an operator on the RV Y mapping into the domain of real numbers, desirably into [0; 1] (see Section II-B).Thus, the index considers the QoE values of all users in the system.

B. Desirable Properties of a QoE Fairness Index
As mentioned in the introduction, the most commonly used quantification of (QoS) fairness is the Jain's fairness index [4].It was designed to have the following properties: (a) Population size independence -it should be applicable to any number of users.(b) Scale and metric independence -the unit of measurement should not matter (for QoE this means independent of L and H values). (c) Boundedness -it should be bounded between 0 and 1 (the index, not the x i values).(d) Continuity -it should take continuous values and changes in resource allocation should change the index (e.g.maxmin ratio is not since it considers only the max and the min, and not values of x i in between).(e) Intuitive -it should be intuitive: high value if fair (index=1 is "perfect" fairness), and low value if unfair (index=0, if possible, is totally unfair).When we want to quantify the QoE fairness we need an index with the same properties as above, plus the following: (f) Deviation symmetric -it should only depend on the absolute value of the deviation from the mean value, not whether it is positive or negative.(g) QoE level independence -fairness is independent of QoE level, whether system achieves good or bad QoE.(h) Valid for multi-applications -should reflect the crossapplication fairness (and not only between users of the same application). 1

C. Rationale of QoE Fairness Index 1) Maximum Standard Deviation of Y:
There exists a maximum standard deviation of the QoE value Y over the bounded value domain [L; H ]. The maximum σ max is obtained when 1 2 of the users experience L and H , respectively.In that case, the average QoE value ( = MOS, as users are experiencing different conditions) is E[Y ] = (L + H )/2 and the maximum second order moment is E Y 2 = (L 2 + H 2 )/2.Then, the maximum standard deviation is (2) 1 The property (h) mainly depends on the QoE mapping function.If Q is able to map QoS parameters to QoE for various applications so as to have comparable QoE values, then our proposed index has this property, too.
2) Jain's Fairness Index, J: The well-known Jain's fairness index can be applied to QoE values However, the coefficient of variation c = σ/μ strongly depends on the expected value E[Y ] = μ.We consider the maximum possible standard deviation for given μ.It is as discussed in [10].
The maximum fairness J max = 1 is reached for the minimum standard deviation (σ min = 0).If we consider the maximum standard deviation σ max we will expect that the fairness index reaches its minimum.Substitute E[Y ] and E Y 2 as given above Eq.( 2) in Eq. ( 3), then This means that the minimum value of J depends on the bounds of the value range [L, H ]. On a 5-point MOS scale with [L = 1; H = 5] then J min = 9/13 = 0.69.Using normalized QoE values with [L = 0; H = 1] then J min = 0.5.
3) New Fairness Index, F: The proposed fairness index, F, needs to fulfill the desired properties above.The standard deviation σ of the QoE values Y quantifies the dispersion of the users' QoE in a system.We define the fairness index F as a linear transformation of the standard deviation σ of Y to [0; 1].The observed σ is normalized with the maximal standard deviation σ max and measures the degree of unfairness.Hence, the difference between 1 (indicating perfect fairness) and σ/σ max is defined as the fairness index.
This can be interpreted as (I) normalizing the standard deviation by the maximum possible standard deviation (1− σ σ max ), or (II) normalizing the QoE values The maximum fairness F max = 1 is for minimum standard deviation (σ = 0).The minimum fairness F min = 0 is found when standard deviation is at its maximum.
In the list of properties it is stated that the fairness index should be intuitive.This means for instance that the index should be at its minimum when the users' experience (at least their scores) is maximally different.This holds for F, but not for J since this depends on the values of L and H . Table I shows the intuitive interpretation of F, while J is not straightforward and violates the following properties: (f) 'deviation symmetric', see Fig. 1; (g) 'QoE level independence', see Eq. 3; (b) 'Scale and metric independence', see Fig. 2b.
Definition 1: The QoE Fairness Index F is defined as the linear transformation F = 1 − 2σ H −L over the QoE Y of all users consuming a service.A system is absolutely QoE fair when all users receive the same QoE value.
We observe that the definition proposed fulfills the properties outlined in Sec.I.Note that F reflects the system perspective of fairness and quantifies fairness of the entire system across all users.It must not be confused with standard deviation of user ratings in a subjective study, i.e. the user rating diversity.The proposed fairness metric requires a clearly defined and bounded value range [L; H ] of the QoE mapping function Q in Eq. ( 1).F is unsuitable to unbounded interval metrics which is however not relevant for QoE assessment.
III. MEASUREMENT STUDY: WEB QoE We consider a simple web page download scenario to highlight in an intuitive and illustrative way the key differences between Jain's fairness index and the proposed QoE fairness index.We conducted a study where n users where downloading the same web page in parallel over a bottleneck link with capacity C = 1 Mbps.A script was implemented which used wget without caching for downloading and started several background jobs at an interval of t = 500 ms at the same host.The incoming and outcoming bandwidth was throttled to C = 1 Mbps.Thus, the n web page downloads shared a common bottleneck.The web page consisting of the main object and all inline objects such as images and style sheets had a total data volume of 753 kB.The measurements were repeated R = 30 times and we obtained for each user i of the n users in run r the page load time x i,r .Measurement methods for obtaining the page load time (PLT) are described e.g., in standards by ETSI [2] or ITU-T [12].
The QoE mapping function Q maps a PLT t to a MOS score Q(t) in the range L = 1 to H = 5.In [13], a logarithmic relationship was observed between t and MOS based on subjective user studies Q(t) = −a log(t)+b .In our numerical results, we use a = 1.5 and b = 4.25 as realistic parameters for browsing through a photo album [13].
Figure 2a depicts the QoE behavior of the system depending on the number n of concurrent users.In particular, the averages over the R runs are plotted for the different statistical measures.We observe a linear increase in the average PLT as well as in the standard deviations of the PLT.Accordingly, the average QoE of all users in the system is decreasing, but in a non-linear way due to the logarithmic relationship.
Figure 2b illustrates the different fairness indexes.Jain's fairness index J is thereby computed for the PLT and the QoE results.The original MOS values 4 and J is computed for both scales.While J leads to completely different results and shapes for Y and Y * , our fairness metric F properly reflects the variances in QoE independent of the scale.This shows again that J is sensitive to the used rating scale in contrast to F. 2 We further observe that there is a strong discrepancy between QoS fairness (using Jain's fairness index (J : PLT) of download times converging to 0.7) and QoE fairness.

IV. CONCLUSIONS AND FUTURE WORK
In this letter we argue that commonly used QoS fairness metrics such as Jain's fairness index are not suitable for quantifying QoE fairness.We have introduced a novel QoE fairness index, and have illustrated how the index fulfils a number of desired properties, which on the other hand are not all fulfilled by using Jain's fairness index.
The application of the QoE fairness metric is manifold, ranging from QoE management mechanisms and system optimization to benchmarking when comparing different resource management techniques.For QoE optimal system design, QoE fairness may be one objective to consider in the optimization problem (e.g., maximize overall QoE across multiple users while achieving a desired level of QoE fairness).QoE fairness says nothing about how good the system is and thus needs to be considered together with the achieved (e.g.mean) QoE in system design.
We want to emphasize that the proposed QoE fairness index F is just a means for doing so.A crucial part is having a proper QoE model Q.Consider e.g., the desired property to be valid for a multi-application scenario.It may be difficult to map QoS parameters to QoE for various applications in such a way as to have comparable QoE values, which is however currently approached in standardization [14].If Q achieves this, then F has this property as well.The proposed fairness index is a generic concept which may also be applied to other domains than QoE (like QoS) or for different purposes.For example, in subjective user studies, we may directly measure QoE and compute the fairness index without a mapping function.In future work we aim to provide a more in-depth investigation of complex service scenarios and the exploitation potential of the proposed QoE fairness index.

TABLE I ILLUSTRATION
OF JAIN'S J AND QOE FAIRNESS INDEX F FOR VARIOUS SCENARIOS AND THEIR DISTRIBUTIONS Y , L = 1, H = 5 Fig. 1.In this scenario, a ratio of p users experience 2 and 1 − p experience 2 + δ.We observe that J violates property (f) 'Deviation symmetric' and is less sensitive than F.