Defining categorical reasoning of numerical feature models with feature-wise and variant-wise quality attributes

Automatic analysis of variability is an important stage of Software Product Line (SPL) engineering. Incorporating quality information into this stage poses a significant challenge. However, quality-aware automated analysis tools are rare, mainly because in existing solutions variability and quality information are not unified under the same model. In this paper, we make use of the Quality Variability Model (QVM), based on Category Theory (CT), to redefine reasoning operations. We start defining and composing the six most common operations in SPL, but now as quality-based queries, which tend to be unavailable in other approaches. Consequently, QVM supports interactions between variant-wise and feature-wise quality attributes. As a proof of concept, we present, implement and execute the operations as lambda reasoning for CQL IDE - the state-of-the-art CT tool.


INTRODUCTION
The features of Software Product Lines (SPLs) and their relationships are represented as Variability Models (VMs) [25], being Feature Models (FMs) the de facto standard. How to include quality information in these VMs, and how to perform quality analyses of SPLs, are two of the main challenges that remain open. Unfortunately, the lack of a broad consensus about how to embed Quality Attributes (QAs) in a VM has led to entirely different solutions, which in most cases are not materialised in a tool [18,32].
Consequently, quality-aware reasoning (e.g., cost or energy consumption) is not directly supported by current SPL solvers and approaches, and hence, the literature eludes this issue [2]. Many workarounds assume that all QAs can be represented as attributes linked to single features (i.e., feature-wise approach) and that the product quality can be computed with simple equations such as addition [10]. Or, rather, they use a combination of solvers connected with external applications such as databases storing QA values or use domain learning procedures to predict the quality of configurations [15], all separate from the VM. All these works propose non-unified solutions, are not efficient, and with limited reasoning capabilities [21]. Most importantly, QAs whose measures (e.g., the energy consumption of a piece of code) depend on many features (e.g., CPU, OS, programming language), cannot be quantified at the VM feature level and need to be measured at the configuration level, i.e., they are product variant-wise QAs [26]. Indeed, in [26] authors claim that using feature attributes to store QAs is only valid for feature-wise QAs (e.g., the Cost associated with a feature). So, VMs and analysis operations should include quality information supporting both feature-wise and product variant-wise approaches. Unfortunately, state-of-the-art tools do not adequately support this yet.
The SPL community need to achieve a formal representation that unifies VMs and quality models (QMs), extensible and fully supported by an automated reasoner. With this objective, we leveraged the modelling and reasoning flexibility of Category Theory (CT) which captures the common aspects of seemingly dissimilar algebraic structures [1]. We published a preliminary work at [19]: an open-source SPL framework that unifies numerical VMs (i.e., supports both boolean and numerical features) with QAs related as Consequently, in this work, we are defining the new Quality Variability Model (QVM), which directly supports quality-reasoning operations on SPL configurations. Extending the published proposal with Section 2, we now allow developers to perform richer variability analyses focused on quality, using common operations on VMs.
Having reviewed the state-of-the-art tools and literature in Section 3, we identified six basic operations that can be combined and extended into more complex ones. Two of them (satisfiability and counting) are considered essential self-analysis operations over VMs. The remaining ones (filtered search, limited search, randomise and an operation combining feature-wise and variant-wise interactions) are standard operations for generating configurations, which we reformulated to include QAs-related queries to be resolved natively by a CT framework (e.g., count the number of configurations within an energy consumption rate interval without listing them, or generate ten configurations with an energy consumption lower than a particular amount).
The main contribution of this work is that these operations can natively analyse the functional variability and its correlations with the non-functional (i.e., quality) requirements for a VM enriched with quality measures, i.e., our QVM. In the proof of concept of Section 6, we first present our lambda-function implementation for the CT editor Categorical Query Language Integrated Development Environment (CQL IDE) to show its potential for SPL stakeholders. CQL is a user-friendly IDE that allows categories to be defined as functional programming databases which are faster and more flexible than classical databases [6]. Then, and for illustrative purposes, we analyse a full scenario of a use case from the intelligent network and edge computing domain, specified through a variability model of network functionalities (i.e., Virtual Network Functions or VNFs) enriched with quality measurements of energy footprint and monetary cost. Additionally, we measured reasoning times of our implementation in CQL IDE. Its results indicate that CQL IDE flexibility and scalability for modelling and reasoning are promising as a tool for both the CT and the SPL communities.

QUALITY VARIABILITY MODEL CATEGORY
This section provides some background on the fundamentals of CT and present the unified Quality Variability Model (QVM).

Fundamentals of Category Theory
Category Theory (CT) is an algebraic theory of mathematical structures [1]. It allows to capture and relate similar aspects of structures while abstracting from the individual specifics of their dissimilarities. A category C represents spaces as a collection of objects related to one another via arrows (i.e., morphisms). Two examples are the categories V , where the objects are vector spaces and the arrows are linear maps, and S where objects are sets and arrows are functions from one set to another. The main concepts of CT are: • Object: a structured class ∈ Ob(C), graphically depicted as a node • . • Arrow: a structure-preserving function ∈ Arr(C) with source and target objects = ( ) and = ( ), respectively, depicted • − → •.
Also, we shall introduce algebraic data integration CT concepts [6]: • Path: a finite sequence of composed arrows: where is a select "unit" object. • Instance: a set-valued functor Inst that assigns values to elements.

Unifying Variability and Quality in a Categorical Model
In [19], we presented a preliminary SPL framework that unifies numerical VMs with QAs as a category where features and QAs are objects, and data-types, hierarchical relationships, and quality and feature constraints are arrows. We now extended that model to provide an alternative framework to abstract any QVM as a category QVM. This framework allows QAs modelling at the feature-wise and also variant-wise levels. Hence, by supporting varian-wise QAs we solve the known challenge of complex QAs with many feature interactions -unfeasible to accurately distribute feature-wise by managing a complex aggregation function. The framework and its equivalences are graphically represented in Figure 1, being the basis for the formalisation of the quality-aware operations further presented in this paper. Concretely, QVM comprises 3 data-type objects (i.e., B for Boolean, Z for integer, and S for characters sets) and 5 structured objects: (1) F : It hosts the extended VM as arrows F → S for the name and domain, F → B for optionality, F → Z for a numerical value, F → F for hierarchical (i.e., Parent) and cross-tree constraints, and additional arrows depending on the number of QAs at the feature-level.
(2) Q : It hosts the quality model as the arrows Q → S for quality name and domain, Q → Z for a quality value, and Q → Q for hierarchical and quality self-constraints.

RELATED WORK
In this section, we summarise the related work that has motivated this work. We list and compare academic and industrial tools, focusing on their support for variability modelling, quality modelling and automatic reasoning. Then, we discuss a motivating usage scenario to illustrate the shortcomings of existing works and the benefits of our approach. Variability Modelling. Its first formalisation dates back to Feature-oriented Domain Analysis (FODA) [16], which proposes Feature Models (FMs) to specify a product family commonality and variability, and external solvers to automatically generate the product variants. FMs are represented as a rooted tree graph consisting of features such as Boolean variables and relationships (see Figure 1, left).
Relationships among features are specified in propositional logic, including tree (e.g., And, Or) and cross-tree constraints. Any SPL approach should provide at least the basic features and constraints defined in FODA. However, more than 45 extensions have been proposed for different needs [7]. The most relevant ones for our approach are: numerical features [20] that are features that can take integer or real numerical values, attributes [4] that allow associating a set of quality attributes to a feature, and complex constraints involving numerical features, attributes, etc. [14] concluded that current SPL tools do not support all the FM extensions and this prevents the adoption of them by SPL development processes.
As discussed in Section 2, the use of categories in our approach simplifies incorporating these extensions [19]. In this paper, we will show that QVM supports all the variability modelling extensions and a set of quality-aware operations.
Quality Modelling. There is no consensus on how to enrich variability models with quality information to reason about the quality of configurations [14,30]. Typically, an SPL engineer would like to obtain the configurations with a quality attribute below a threshold (e.g., get configurations that consume less than 3 Joules), or to generate the best-qualified configuration (e.g. trade-off between energy consumption and performance). In summary, we consider two different approaches -one that extends traditional FMs, either with attributes or with an FM sub-tree, and one that defines a quality model and links quality measures with configurations.
In the first approach, quality information is provided at featurewise level by attributes -i.e., each feature contributes individually to the system QA. In this case, quality measures are linked to single features using attributes [4] (e.g., the WiFi feature consumes 3 Joules). To compute the quality of feature configurations an aggregation function is defined, such as addition, minimum or maximum [26]. This approach is supported by Clafer, AAFM Python framework [8] (and the formerly FAMA), FeatureIDE, pure::variants and SPL Conqueror, all described in [14]. The most advanced extension formalisation is the Universal Modelling Language (UVL) [29]. However, it does not consider variant-wise QAs, nor there mentioned state-of-the-art solvers are UVL ready.
An alternative is to extend the FM by incorporating QA-specific features in a subtree. This is the case of QAMTool [33]. But [26] demonstrates that using attributes is only valid for feature-wise QAs, such as footprint or cost, where we can either measure a single   [27], mainly due to not considering the feature interactions.
However, even defining better distribution algorithms, the problem remains for those QAs, such as performance and energy consumption, that do not make sense to measure at the feature-level (e.g., the energy consumption of WiFi feature depends on other features like data length, package size, distance to the access point, etc.). For these QAs, it is impossible to quantify their influence on individual features because the measurements have meaning only for a concrete configuration variant. These QAs are classified in [26] as variant-wise QAs, and several works consider that the set of measurements of a QA should be univocally linked to a configuration variant [9,11,13].
In some cases, an external quality model is defined (e.g., a goal model); and the QAs measurements are usually linked to the configurations through a database.
In [19], we highlighted the benefits and drawbacks of approaches to defining an external quality model, with two important conclusions: (1) most existing solutions are not directly compatible with automated quality-reasoning, and (2) SPL reasoning lacks a "unified" model that appropriately supports quality metrics. Sometimes these two approaches (i.e., feature-wise and variant-wise) are combined, as in QAMTool that uses the NFR framework [32] to externally represent QAs in a goal model and also an FM subtree to include quality information as part of the tree.
Our approach QVM is representative of the latest approach defining a unified model using a category theory framework with native support for quality reasoning. Also, QVM supports both the feature-wise and variant-wise approaches by associating at the configuration level the result of the feature-wise aggregate function. To the best of our knowledge, there is currently no proposal providing quality-aware reasoning of product variability, similar to QVM.
Automatic reasoning All tools offer some level of automatic reasoning. Some provide their implementations of all or a subset of the operations for self-analysis and basic generation of configurations defined in [3]. Others use a third-party modelling language that provides such support.
The number of provided operations considerably depends on the support provided by solvers. Regarding quality-aware operations, current approaches do not natively support the full set of qualityaware operations due to different reasons: incompatibility of SAT solvers with non-Boolean features such as quantified QAs [14], limitations of Constraint Programming (CP) solvers, which do not make assumptions on the algebraic properties of a solution space [12], limitations of the modelling language like the feature-level QAs in MOO Clafer tool [23], intrinsic complexity to adapt or extend the specific searching algorithms like counting in Satisfiability Modulo Theories (SMT) solvers [5], and/or scalability and other reasoning issues like in the extended VMs [22].
For instance, only Clafer, FeatureIDE and SPL Conqueror allow sampling configurations, and the process is not entirely random [20]. Moreover, these tools use quality information only to optimise configurations and to calculate the overall impact on a QA. Thus, the main contribution of this paper is the uncovering of a qualityaware version of the operations typically defined for a VM, more than those typically supported in other approaches, as part of a unified QVM.

MOTIVATING USAGE SCENARIOS
We now present a usage scenario in an EDGE case study.
As shown in Figure2.a), the system comprises 3 different configurable network interfaces (namely Mobile, WiFi and Ethernet), 3 sizes of Antennas, and 3 different Virtual Network Functions (VNFs) (namely Monitor, Firewall, Encryption). EDGE contains the cross-constraint that Ethernet excludes Antenna, since it is a wired system. Alice can select a VM approach discussed in Section 3, but only those supporting numerical features, attributes and complex constraints.
But Alice is also interested in performing certain quality analyses based on  • She can easily integrate the Cost measurements into the variability model as features' attributes because Cost is measured feature-wise (see Figure2.a)). • However, she cannot do the same for the Runtime and Energy Rate, which are measured at the configuration level. This implies that she has to design, implement and use an ad-hoc external mechanism (e.g., a DB) to store these measures and link them to configurations in the FM (Figure2.b).
Scenario 1.2. Alice decides to use our approach: • We are offering Alice an integrated approach that will allow her to model variability with numerical features and complex constraints and, more importantly, to store both feature-wise and product variant-wise QAs using the same framework (Figure 3).
However, although relating QA measurements and VMs is important, what Alice really needs is a list of quality-aware operations to perform the quality-analysis of her system. For instance, she first wants to check that the variability model is correct and that the reasoner can produce solutions that have Energy Rate and Runtime QAs information and thus she needs a SATISFY operation with regards to QAs. Then, she wants to count configurations, and not just the number of valid ones, but the number of configurations measured for both Runtime and Energy Rate values, needing a COUNT operation that considers QAs. Further, she wants to have a first idea of the kind of features and performance behaviour of EDGE instances and wants to bound the number of configurations to 5 configurations with Energy Rate, using for that a BOUND_RANDOM operation with QAs. This guides her to realise that the company only works with mobile networks interfaces, so the WiFi and Ethernet alternatives are meaningless.
As a consequence, she wants to filter the VM, having Mobile feature as a requirement (FILTER operation). She notices that the QA cost is indicated only at the feature level and that there are too many solutions yet. Consequently, she runs FILTER with constraints Mobile and not 3G, alongside AGGREGATE with the sum-aggregated function for Cost. At this point, she notices that some configurations are too costly for the performance that they provide. Finally, she runs again FILTER_AGGREGATE with an additional constraint: (Cost ≥ 40 $) ⇒ ((Energy Rate ≤ 18 Watts) ∧ (Runtime ≤ 10 Seconds)). The solver returns several instances, showing her that a 5G network with a large antenna is optimal, besides for the Monitor virtual function, in which 4G networks with a small antenna behave equally with a fraction of the cost.
In order to perform the kind of analysis described above, there are two options: Scenario 2.1. Alice decides to use existing solutions: • Since operations implemented using current solvers (SAT-ISFY, COUNT, FILTER, ...) do not take into account QAs, she has to provide her implementations of quality-aware operations (Figure 2.b)). • In her implementations she needs to consider the different representations for feature-wise and variant-wise QAs.
Scenario 2.2. Alice decides to use our approach: • We are offering Alice a list of defined and implemented qualityaware operations using the native support provided by the framework used to model variability and QAs. We provide more of its details in the next section.

QUALITY-AWARE REASONING OPERATIONS
In this section, we identify the basic operations to reason over QAs variant-wise. More complex operations can then be formed by combining and extending the basic ones. To illustrate these, we provide examples based on the instantiated EDGE category of Figure 3, which is graphically presented following the olog standard -the CT knowledge representation framework for real-world systems [28]. EDGE is our standard case study and comprises the categorical QVM of the edge-computing scenario described in Section 4.

Basic Operations on Quality Variability Models
We shall focus here on the operations typically needed to perform advanced reasoning over QVMs.  [20,24]: Randomise the generation of a QVM measured solution space for any operation. For example, generate EDGE measured configurations starting from a random seed. (6) Aggregate, AGGREGATE: Transform QAs at the feature level into QAs at the configuration level based on aggregation functions. E.g., the aggregated value of the cost of an EDGE configuration is the sum of the leaf features' costs.

IMPLEMENTATION IN CQL IDE AS LAMBDA FUNCTIONAL ALGORITHMS
In this section, we first provide details of our implementation of the six formalised QVM operations. They are presented as functional algorithms based on lambda functions. For our implementation, we chose a state-of-the-art tool to model and reason over categories: the Categorical Query Language (CQL) IDE 1 . CQL is a functorial language used for functional programming with lambda functions; for background details, we kindly point the readers to [17]. 1 CQL IDE main website: https://www.categoricaldata.net/ Some details of CQL are: • Basic data types and functions are defined as element type arrows between objects. • A structured category is a schema of objects, and element or path types of arrows. • A functor is a query over a source schema to a RES schema.
• A literal instance generates variables and assigns the values.
• The reasoning is an eval instance of a schema literal. If any reasoning is undefined, the IDE halts and informs of the error; for instance, that CC is not functorial due to a non-existing referenced feature. Otherwise, CQL IDE correctly computes what is specified in the instances and queries.
We now follow by presenting an overview of our implementation in CQL IDE of the identified QVM operations. We start with the Algorithm 1 implementing the SATISFY operation. Algorithm 1 is a single lambda function that returns True if any configuration (i.e., ) and QA (i.e., ) relationship exists. As it is only computed over the QMC object, it completely abstracts from features or QAs names and values. Algorithms 3 to 5 share some code, as they generate configurations with their respective QA values. Hence, in the three algorithms, besides QMC, we also need CC and QA objects as inputs. On line 1 we initialise the result as the empty set Res, which is then filled in a forall loop iterating over each instance (i.e., relationship) of QMC. Due to the if clause on line 3, just configurations related to QAs and vice-versa are considered in each iteration. Two lambda functions are filling Res. The first one is on line 4, and it retrieves the features associated with the configuration of a specific iteration. The second one is on line 5, and likewise retrieves the data of the set of qualities of a specific iteration. Finally, in each iteration, the features and QAs are copied to Res on lines 6 (7 for aggregation).
Continuing with the differences, for FILTER in Algorithm 3 we need to consider an additional input in the form of a set of features or quality constraints, which are then considered in the if clause of line 3 to filter the selected configurations and QAs. In Algorithm 4 we merged BOUND and RANDOM operations forming a random sampling. For that, we find two new inputs, a Limit (L) of solutions and a random Seed (S). On line 3, L restricts the number of tested loop iterations, and on Line 2 QMC is shuffled based on S.
Finally, in Algorithm 5 we have an extra input in the form of an aggregation Function (F) for QAs at the feature-level (e.g., addition . Consequently, F is transformed into its correspondent lambda function on line 6, and adds additional calculated QAs to the results. This algorithm shows how feature-wise and variantwise QAs can be treated uniformly in our approach. Notice that running the AGGREGATE operation before any of the other ones allows disposing of the feature-wise information at the configuration level. After testing them by running CQL IDE, its reasoning time averages to 0.13 seconds for Figure 3 QVM. Consequently, we guess that CQL IDE is scalable due to its characteristic combination of reasoners: an automated theorem prover with Knuth-Bendix completion, and hashing, balanced binary search trees, and chasing algorithms. This selection comes from different fields: formal methods, SAT solving and relational algebra respectively.

PROOF-OF-CONCEPT
In this section, we revisit the usage scenario of the EDGE case study presented in Section 3 to show how Alice uses our approach to perform the QAs analysis of her EDGE scenario. First, Alice wants to check that the QVM is correct and that the reasoner can produce solutions. She runs SATISFY, and the CQL IDE returns an instance of RES with the value True, implying satisfiability without reasoning ambiguities.
Then, she runs COUNT to learn how many configurations the system has. CQL IDE returns an instance of RES with the value 63. Further, she wants some idea of the kind of features and performance behaviour of EDGE instances, she runs a BOUND_RANDOM of 5 configurations. Since her company only works with mobile interfaces, she runs FILTER selecting the Mobile feature. CQL IDE returns 36 instances of RES comprising the same number of configurations with 2 valued QAs each, runtime and energy rate.
Since the QA cost is indicated only at the feature level, there are too many solutions. Consequently, she runs FILTER with constraints Mobile and not 3G, alongside AGGREGATE with the sum-aggregated function for cost. CQL IDE returns 27 instances of RES comprising the same number of configurations with the respective values of the 3 QAs -runtime, energy rate and cost.
As she notices that some configurations are too costly for the performance that they provide, she reruns FILTER_AGGREGATE with an additional constraint: (Cost ≥ 40 $) ⇒ ((Energy Rate ≤ 18 Watts) ∧ (Runtime ≤ 10 Seconds)). CQL IDE returns 6 RES instances, showing her that a 5G network with a large antenna is the best option, besides for the Monitor virtual function, in which 4G networks with a small antenna behave equally with a fraction of the cost.

CONCLUSION
Before the unifying CT framework for SPLs [19], variability, quality, and the relationship between configurations and valued QAs have been modelled separately. This paper highlights the lack of methods and tools designed explicitly for QAs values at the feature and configuration levels. To address this, we identify the basic operations to quality-reason over QVMs: satisfiability, count, filter, bound, randomise, and aggregate.
Then, we present the categorical functional algorithms based on lambda functions for QVM . They are a guidance for CT reasoners implementation based on queries to a category. These operations over QVM mainly operate on its object of binary relationships QMC, as it provides complete configurations with valued QAs pairs. Additionally, we show how to combine operations to form complex ones. Further, we implement them in CQL IDE and perform a performance test for EDGE QVM with several casuistic.
CQL IDE QMC suggests scalability and hence a promising tool for the modelling and reasoning SPLs. We end with a usage scenario analysing the use of our approach in the EDGE case study.
In future work, we plan to properly perform a scalability test with large QVM and more advanced queries (e.g., guided sampling, prediction). In addition, we are considering QVM implementations in other CT environments such as Haskell.