Aggregate Metric Model for Evaluating Business Processes

Purpose - The purpose of this paper is to present a business process measurement framework for the evaluation of a corpus of business processes modelled in different business process modelling approaches. The results of the application of the proposed measurement framework will serve as a basis for choosing business process modelling approaches. Design/methodology/approach - The approach uses ideas of the Goal Question Metric (GQM) framework to define metrics for measuring a business process where the metrics answer the questions to achieve the goal. The Weighted Sum Method (WSM) is used to aggregate the measure of attributes of a business process to derive an aggregate measure, and business process modelling approaches are compared based on the evaluation of business process models created in different business process modelling approaches using the aggregate measure. Findings - The proposed measurement framework was applied to a corpus of business process models in different business process modelling approaches and is showed that insight is gained into the effect of business process modelling approach on the maintainability of a business process model. From the results, business process modelling approaches which imbibed the principle of separation of concerns of models, make use of reference or base model for a family of business process variants and promote the reuse of model elements performed highest when their models are evaluated with the proposed measurement framework. The results showed that the applications of the proposed framework proved to be useful for the selection of business process modelling approaches. Originality - The novelty of this work is in the application of WSM to integrate metric of business process models and the evaluation of a corpus of business process models created in different business process modelling approaches using the aggregate measure.


Introduction
Business process measurement enables organizations to improve their performance since the measurements evaluate the efficiency of business processes and so suggest improvement to such processes.The measurements of business processes rely on structural metrics which measure the attributes of a business process such as the number of nodes, the number of split constructs etc. (Vanderfeesten et al., 2007).Some attributes of a business process model such as understandability and modifiability which belong to the more general concepts of maintainability correlate with many metrics of the business process model (Sánchez-González et al., 2010) and stakeholders have different preferences of metric for measuring these attributes (Yen, 2009).There is a lack of consensus amongst stakeholders in organizations and authors on the metrics for the measurement of a business process (Rolón et al., 2006;Yen, 2009).Some of the important qualities which organizations want from a business process such as maintainability, understandability, modifiability and usability etc. have many attributes.Therefore, it is necessary to have a measurement framework to derive a measure that reflects the different attributes (Azim et al., 2008;Sánchez-González et al., 2010).
The individual metric for measuring business processes are not adequate on their own to measure quantities which have many attributes, which is a knowledge gap that needs to be filled.The need to measure quantities with multiple attributes arises in a situation of modelling a family of business processes.The modelling of a family of business process variants can be very challenging because when variants are modelled separately, it leads to duplication of nodes, resulting in redundancies and inconsistencies.Meanwhile, consolidating all variants in one monolithic model, which is a superset of all variants in a family leads to less redundancy but more complexity of the model and therefore hindering comprehensibility of the model (Milani et al., 2016).In this situation, it becomes necessary to derive a measure for the trade-off between the redundancy and complexity of the business process model.
We proposed a measurement framework called Aggregate Metric Model (AMM) for evaluating a corpus of business processes create in different business process modelling approaches where the metrics of the attributes are aggregated to derive a single measure.
The objective of this research is to: -Derive aggregate measures of a business process model that reflects trade-offs of different metrics of a business process model.-Derive measure that reflects the stakeholders' different preferences on the metric for business process models.-Enable the evaluation of business process models in different business process modelling approaches to compare business process modelling approaches.
In this paper, we assume that the metrics which make up an aggregate measure of a business process are given, and each one of the metric correlates with the goal or attribute which the aggregate measure seeks to measure.
This paper is organized as follows.Section 2 elaborates on previous related work on the measurement of business processes.Section 3 describes our business process measurement framework.In Section 4, we describe the application of our business process measurement approach and the results obtained.Finally, Section 5 recaps our contribution and give the limitation of our business process measurement framework.

Related Work
In this section, we review business process measures to identify any gaps in existing research to suggest areas which require further research.

Measurement of Business Processes
Business processes are measured to identify areas that organizations need to improve.The attributes of the business process are measured to give insight into how the business process fulfil the goals set by an organization.Several business process metrics are adapted from software engineering because Cohesion: is a measure of the strength of the relationship between operations in an activity of a business process model.A strong relationship between the operations of activity is mostly desired (Reijers and Vanderfeesten, 2004).Reijers and Vanderfeesten, (2004) presented a metric for measuring the cohesion of operations in activities of a business process.
Coupling: it is the measure of the degree of interdependence between the activities, which is a description of how the activities in a business process model are connected.It is calculated by counting all pair of activities in a business process model.Loosely coupled business process models are mostly desired (Vanderfeesten et al., 2008).
Entropy: The entropy of business process models is a measure for quantifying the uncertainty of process execution (Jung, 2008).
Complexity: Complexity measures the simplicity and understandability of a model (Cardoso et al., 2006).
Flexibility: Flexibility is the capability of a business process changing, and it can be classified by the dimension of change (Regev et al., 2006).There are three orthogonal dimensions of change: the abstraction level of change, the subject of change, and the properties of the change.Abstraction changes are the change of business process model or change of business process instance.The subjects of change deal with the perspectives of change.The different perspectives can be found in any of the abstractions of change.There are five basic perspectives, the functional perspective (change in the goal of a process), the operational perspective (change in the activities), the behavioural perspective (change in the sequence flow), the informational perspective (change in the data objects) and the organizational perspective (change in roles, e.g.ownership and permissions).There are four properties of change: the extent (can be incremental or revolutionary), the duration (temporary or permanent changes), the swiftness (immediately or deferred) and the anticipation of change (planned or ad hoc) (Regev et al., 2006).The definitions for business process flexibility are mostly qualitative measures, and there is not much research in the quantitative measure for flexibility except for the quantitative measure of business process flexibility introduced by Li, Reichert and Wombacher (2008) which is the distance between a process model and its variant.This distance is the minimal effort or number of change operations (delete, move, insert) on activities required to transform a process model to a variant (Li et al., 2008).

Measurements of Business Process Execution
Business process execution measures quantify how the process is executed at run time, and they are related to the dynamic properties of business processes.Measures concerning execution can be used to compare results with expected results to improve customer satisfaction (Sánchez González et al., 2010).Some attributes of business process execution that exist are execution time, quality, usability, reliability, functionality, cost, and effectiveness (Sánchez González et al., 2010).Most of the attributes for business process execution such as quality, usability, reliability, functionality, effectiveness etcetera are qualitative and lack metrics for measurement, which may be because these measures of business process execution seek customer satisfaction and as such the results of business process execution have been studied in business-related sciences and not in computer science (Sánchez González et al., 2010).For instance, the execution time of many models cannot be measured because they are created with business process modelling formalisms that are not executable, or the business process modelling formalisms exist only in theory (La Rosa et al., 2017).As a consequence, such business process models lack execution engines to support the execution of business process models.

Interdependency of Attributes and Their Relative Importance
Some attributes of a business process are attributes of other attributes as illustrated in Figure 1 where ovals represent the attributes.The arrows pointing to an attribute indicate the attributes it has and the number of attributes pointing to an attribute indicates the importance of an attribute as a measure of a business process.The business process interdependency map shown in Figure 1 borrows ideas from the visual project mapping introduce in (Killen and Kjaer, 2012).Understandability and changeability of a business process model are recognized as attributes of maintainability (Canfora et al., 2005).The complexity of a business process is an attribute of understandability and changeability and by extension attribute of maintainability (Azim et al., 2008;Cardoso et al., 2006;Rolón et al., 2006).The redundancy of nodes in a business process model has an inverse correlation with the maintainability and changeability of a business process model because duplicate nodes affect the readability of a model and effort required for modification of nodes (Koehler and Vanhatalo, 2007).Milani et al. (2016) agree that the more redundancy in a family of process variants, the more difficult it becomes to maintain such a family of variants because as processes evolve, any change required for an activity will have to be applied to all the duplicates of such activity node (Milani et al., 2016).Flexibility correlates with maintainability and changeability because it has to do with the ability to change the business process.Therefore, flexibility is considered as an attribute of maintainability and changeability (Azim et al., 2008;Canfora et al., 2005).
In the study carried out by the authors (Sánchez González et al., 2010), they showed that understandability and changeability are the measurable attributes of the business process that most business process metrics seek to measure.In the experiment conducted by (Sánchez-González et al., 2010), they stated that understandability significantly correlated with several metrics that measure the number of nodes, gateway heterogeneity, the path from a start node to end and nodes connected to decision nodes.They also stated that modifiability (changeability) has a significant correlation with gateway heterogeneity and the number of nodes connected to decision nodes.Attributes with more arrows pointing to them tend to be at a higher level of detail while attributes with fewer arrows pointing to them tend to be at a lower level of detail and more closely related to the structure of the business process model.Therefore, the attributes at a lower level of detail are more likely measured with metrics related to the structure of a business process model.

Attributes Relevant to The Maintainability of a Business Process Model
Complexity, redundancy and flexibility are the lowest level attributes directly or indirectly related to the maintainability of a business process model, as shown in Figure 1.Therefore, we will use the measures of these low-level attributes to derive the maintainability of a business process model later on in Section 4. In section 4, we will aggregate the metrics for these attribute to gain insight into the trade-offs between them and also gain insight into the maintainability of business process models.Hence we will describe in greater detail redundancy, complexity and flexibility, which can be measured with quantitative metrics as follows.Milani et al. (2016) measure redundancy as the duplication rate of activity elements in all the variant models in a family.So, an activity occurring x times across all variant models counts as x-1 duplicate occurrences.The total duplicate occurrences of all activities in a family of variants are the sum of duplicate occurrences of all activities in the family of variants.Therefore, the duplicate rate is the total duplicate occurrences divided by the total number of all activities in the family of business process models.

Metric for Redundancy
Definition: Let  be a set of all activities contained in a family of variant business process models, where  1 ,  2 , …   are all activities in .
Let  be the total number of activities in all variants involved.
For all activity   in  let (  , ) be the number of times   occurs in .
The duplicate rate () = ()/.The metrics, the total duplicate occurrences of all  activities in , dupl(A) or the duplicate rate, dr(A) can be used to measure the redundancy of activities of business process models.We will use dupl(A) for our evaluation of the redundancy of business process models in Section 4. We will not use the duplicate rate, dr(A), because we will be using the repeated-measures experimental design where a business process is modelled in each of the business process modelling approaches under comparison (Verma, 2015).Another reason for using this rather than using a duplicate rate is that the other attributes, complexity, and flexibility which along with redundancy are attributes of maintainability are not measured as rates.

Metric for Complexity
Most of the metric for measuring the complexity of business process models are adapted from the metric for measuring the complexity of software program.The earliest measure for measuring the complexity of software program was proposed by McCabe (1976), which is known as the MCabe's cyclomatic complexity (MCC).The MCC for a program module is defined as  −  + 2, where e and n are the number of edges and nodes in the control flow graph respectively.Cardoso ( 2005) designed a business process complexity metric call Control-flow Complexity (CFC) that borrows ideas from McCabe's cyclomatic complexity.The advantage of CFC over MCC is that the design of CFC took into account the different semantics of the process nodes (e.g.AND-split, XOR-splits, and OR-splits) (Cardoso, 2005;Cardoso et al., 2006).The main idea behind the CFC was to evaluate the number of mental states that must be considered when a designer encounters a split in a process model.When a split (AND, XOR, or OR) is introduced in a process model, the business process designer must mentally create a map of the number of states that can be reached from the split.The complexity of a process model is calculated by summing the CFC of all split constructs in the model.The complexity for AND, XOR and OR split constructs is defined as follows: -CFC for AND split: since all edges going out from an AND-split must be processed; only one state needs to be considered, which means the CFC of AND-split is 1.Therefore, for every AND-split in a model, 1 is added to the CFC metric.
-CFC for XOR split with s outgoing transitions: exactly one from s possible paths must be taken, so s possible states are considered.Consequently, every XOR-split with s outgoing edges adds s to the CFC metric of this model.
-CFC for OR split with s outgoing transitions: at least one and at most n outgoing edges must be processed when OR-split is encountered, therefore every OR-split with s outgoing transitions adds 2  − 1 to the CFC metric.
Let n be the number of XOR constructs in a model.For all   in a model, the Control Flow Complexity,   , is ∑   =1 Let n be the number of OR constructs in a model.For all   in a model, the Control Flow Complexity,   , is ∑ 2   =1 − 1 Let n be the number of AND constructs in a model.For all   in a model, the Control Flow Complexity,   , is The CFC for a model is calculated by summing CFC's for all split constructs in the model, which is: CFC = ∑  + ∑  + ∑  Control Flow Complexity (CFC) is adopted for measuring the complexity of business process models in Section 4.

Metric for Business Process Flexibility
We adopted the measure of process distance introduced by (Li et al., 2008) for measuring the flexibility of process models.They define flexibility as the minimal number of change operations or effort needed to transform a process model (as-is model) to the desired variant model (to-be model).
We will use the minimal number of change operations (e.g.add, delete or move activity or fragment) for measuring the flexibility of business process models in Section 4 (Li et al., 2008).
The minimal number of change operations can also be viewed as the effort for transforming a business process model to a business process instance where there is a deviation between the schema of the business process instance and the business process model.Even though in theory the minimal number of change operations can also be used as a measurable concept of business process execution, we will not use it for this because we are not sure of what the schema of the business process instances for the different modelling approaches will be.The total number of minimal change operations has an inverse correlation with the flexibility of a business process model (Li et al., 2008).
Each change operation (insert, delete or move) has a cost of 1.When a smaller number of change operations is needed to achieve a transformation, this translates to less effort needed for transformation, which means greater flexibility and the converse applies for a larger number of change operations.Therefore, the minimal number of change operations has an inverse correlation with flexibility.To find the minimal number of change operations needed to transform a process model S = (N, E, . ..) ∈ P into another model S' = (N', E', . ..) ∈ P, where N is the set of nodes, E the set of edges, and P is the set of all process models; three steps are needed: Where a1, a2, . . ., an are activities which are nodes, therefore, N = {a1, a2, . . ., an}.
To transform S into S' perform the following steps: 1. ∀ai ∈ N \N': delete all activities being present in S, but not in S'.
2. ∀ai ∈ N ∩N': move all activities being present in both models to the locations as reflected by S'.
3. ∀ai ∈ N' \ N: insert those activities being present in S', but not in S.
It is easy to determine the number of delete or insert operations, but it is not easy to determine the optimal move operations.So, to solve this move optimization problem, an order matrix An×n with n = |N ∩N' | is needed.Where n is the number of activities present in both S and S'.The order matrix represents the control flow dependency between all pairs of activities in both S and S'.Four types of control relations can be identified in an order matrix defined below: Order matrix: Let S = (N,E, . ..) ∈ P be a process model with N = {a1, a2, . . ., an}.Let trace, t, be a sequence or flow from one activity to another.Let TS denote the set of all traces producible on S. We use t(ai ≺ aj) to denote that ai appears before aj in t.Then: Matrix An×n is called order matrix of S with Aij representing the relation between different activities ai, aj ∈ N: That is, Aij = 1 if for all traces containing activities ai and aj, ai always appears BEFORE aj.
That is Aij = 0 if for all traces containing activity ai and aj, ai always appears AFTER aj.
That is Aij = * if there exists at least one trace in which ai appears before aj and at least one other trace in which ai appears after aj.This means that ai and aj are contained in different parallel branches.
That is Aij = -If there is no trace containing both activity ai and aj.This means that ai and aj are contained in different branches of conditional branching.
The main diagonal of the order matrix is empty since an activity is not compared with itself.Elements Aij and Aji can be derived from each other since if activity ai is a predecessor of activity aj (i.e.Aij = 1), we can always conclude that Aji = 0 holds.Similarly, if Aij ∈ {'*', '-'}, we will obtain Aji = Aij.Therefore, the problem can be simplified by only considering the upper triangular matrix A = (Aij )j>i.Therefore, an order matrix A can uniquely represent the process model on which it was built.An example of an order matrix is shown in Table I, which is built from the process model in Figure 2.
If we compare the order matrices Table I and Table II of two processes shown in Figure 2 and Figure 3 respectively, we see that there are conflicts in the corresponding cell entries of both matrices.Let us look at the formal definition of conflict from (Li et al., 2008).Definition Conflict: Let S, S' ∈ P be two process models with the same set of activities N. Let A and A' be the order matrices for S and S' respectively.Then we say that activities ai and aj are conflicting iff Aij ≠ A'ij, written (,  ′ ) ≔ { (  ,  ) |  ≠ ′  } then corresponds to the set of all existing conflicts.
The conflicts between the order matrices are shown as the shaded cells in the order matrices illustrated in Table I and Table

Table VI Karnaugh map
From the Karnaugh map in Table VI, the following simplification is derived:

DE + EF + CDF
There are two minimal move operations required, which are moving D and E or moving E and F. Therefore, the minimal number of change operations required to resolve the conflict CF2 is 2. We then add together the minimal number of operations required to resolve all groups of conflicts.Since we earlier calculated that the minimal number of operations required to resolve the conflict CF1 is 1, therefore, the minimal number of operations required to transform S1 to S2 is 3.

Measurement Approaches
Due to the many attributes of the business process, there is the problem of lack of consensus amongst authors on the measurement for a business process (Rolón et al., 2006;Yen, 2009).Metrics for measuring business processes adopt a bottom-up manner of measurement where a metric is first defined before specifying the goal of an organization which it seeks to measures (Koziolek, 2008).This bottom-up approach for defining metrics creates the problem of metrics not adequately measuring the goal of an organization.Because of the difficulty in selecting relevant metrics due to lack of EF 00 01 11 10 CD 0 0 1 0 0 0 1 1 0 1 1 1 0 0 1 0 EF DE CDF consensus and the problems of defining metrics for measurement of business processes using a bottom-up manner (Koziolek, 2008), it becomes necessary to have measurement approaches for defining metrics.
One measurement approach that has been applied in software engineering is the Goal-Question-Metric (GQM) (Basili and Weiss, 1984) which can be used for deriving the metrics used for measuring a business process.Another approach is the Integrated Model for Business Process Measurement, which is based on the GQM approach, and it offers guidelines to implement a quantitative analysis of a business process.
In the planning phase, the desired improvement (e.g., maintainability, performance, etc.) of a process is defined.During the definition phase, measurement goals, questions and metrics are defined.The GQM approach adopts a top-down method for deriving metrics for business processes by first specifying the goals, and then questions are asked whose answers will help in attaining the goals.The metrics are defined in a third step to provide a scheme for measuring (Koziolek, 2008).For each goal, an organization wants to achieve, a set of related questions must be asked, and each question can be answered by one metric or a collection metrics.A guideline can be provided to analyze the data derived from the measurements to have an insight into the goal (Azim et al., 2008;Caldiera and Rombach, 1994).Since there are numerous measurements for business processes, one can easily incorporate many, which will result in many data.Therefore, stating goals in advance ensures only relevant metrics for achieving the goals are selected, which helps in reducing the effort needed for data collection (Koziolek, 2008).The benefit of GQM is that each metric serves a purpose and not defined for the sake of measuring (Berander and Jönsson, 2006).
The definition phase of GQM framework has three levels: 1. Conceptual level (goal).A goal an organization wants to achieve is defined for a business process.

Operation level (question).
A set of questions is used to characterize the business process with respect to a selected quality issue.The questions ask for specific quality attributes and making sure that certain aspects of the goals are concrete.3. Quantitative level (metric).A set of metrics is associated with every question.
The actual measurement takes place in the data collection phase, which may be performed manually or electronically.In the interpretations phase, the collected data from the data collections phase is processed to gain measurement results which can be used to answer the questions, and with the answers, it can be evaluated if the goals have been attained (Koziolek, 2008).

The Integrated Model for Business Process Measurement
Another measurement approach is the integrated approach for business process measurement which is based on the GQM approach but what is unique about it is that it is an additive weighted model of the metrics of attributes of a goal (Yen, 2009).The relevant measures with respect to the goal of a business process are combined into one overall measure.The measurement for a goal is multi-criteria because the goals of a business process are meant to satisfy the diverse goals of stakeholders who consist of customers, employees, to name a few.The integrated approach is used to evaluate the trade-off between attributes because of the weighted sum of the measures.The integrated approach is used in a situation when we are faced with a design issue of providing a set of measures of a business process that reflects all stakeholder's interests.The integrated approach has the steps: (a) defining the business goals (b) defining questions and metrics (c) evaluate the outcome of each measure (d) the final step is to calculate the weighted sum of the measures (Yen, 2009).Since the integrated approach for measurement can combine many measures to describe a goal, this way it solves the problem of lack of consensus amongst authors on the measurement for a business process (Rolón et al., 2006;Yen, 2009).

Aggregate Metric Model for Evaluating Business Process
We present an Aggregate Metric Model (AMM) approach, which builds on the GQM and Integrated measurement approaches.AMM aims to evaluate a corpus of business processes designed in different business process modelling approaches.A weighted sum of the measure of the attributes (evaluation criteria) of a business process is calculated by using the Weighted Sum Method (WSM) (Yoon and Hwang, 1995).The weighted sum method (WSM), which is a Multi-Attribute Decision Making (MADM) method is used to obtain the performance score for each business process modelling approach.MADM techniques provide solutions to problems involving conflicting and multiple objective attributes (Pohekar and Ramachandran, 2004).
AMM consists of three phases: definition, data collection and evaluation.

Definition
The desired attribute of a business process which requires improvement, which is most times a goal, is defined along with questions and metrics.In this phase, a model of the attribute mapping is created.The attribute mapping enables us to understand the attribute and their interdependency, which will enable the understanding of the importance of each attribute.The attribute mapping will help in the formulation of goals, questions and metrics.

Data collection
Actual measurement and recording of the attributes of the business processes are performed.The attributes for a corpus of business processes modelled in the different business process modelling approaches under investigation is measured.

Evaluation
The steps for evaluation are: (a) calculate the average of evaluation criteria (b) form a decision matrix of approaches, and evaluation criteria (c) normalize the data in the decision matrix (d) a weighted normalized decision matrix is formed from the normalized decision matrix (e) sum the values for the evaluation criteria for each approach in the weighted normalized decision matrix.The steps for evaluation are described as follows: a. Calculate the average measurements for each evaluation criteria for each of the business process modelling approaches where the measurements for the evaluation criteria are the recorded output of the data collection phase.b.A decision matrix  = {  ,  = 1, 2, … , ;  = 1, 2, … , } of the average measurements from step (a) is formed from the approaches   ( = 1, 2, … , ), which are alternative, and the evaluation criteria   ( = 1, 2, … , ).c.Normalization is used to transform the measurements of the evaluation criteria in the decision matrix from (b) to a compatible unit scale.The linear scale transformation is used by dividing the values of the evaluation criteria by the maximum value for those evaluation criteria (Hwang and Yoon, 1981;Yoon and Hwang, 1995).For benefit criteria, whose higher value is desired, the normalized value   is obtained by

𝑚𝑎𝑥
For the cost criteria, whose lower value is desired,   is computed with Where    is the maximum value for a criterion   ( = 1, 2, … , ) (Chakraborty and Yeh, 2007;Hwang and Yoon, 1981;Jahan and Edwards, 2015;Yoon and Hwang, 1995).In the normalized decision matrix, all the evaluation criteria, which are cost criteria, have their best value as 1 and their worst value as 0, and benefit criteria have their best value as 0 and their worst value as 1. d.A weighted normalized decision matrix is derived from the normalized decision matrix from (c) by applying the following formula to derive the weighted values for the weighted normalized decision matrix Where   is the weight attribute.A weighting vector  = ( 1 ,  2 , … ,   , … ,   ) is added to the normalized decision matrix, and the weighted normalization decision matrix is derived from the normalization decision matrix from (c).The weighting vector W represents the relative importance of m evaluation criteria (Hwang and Yoon, 1981;Yoon and Hwang, 1995).e.The performance scores for each business process modelling approach is obtained by summing the values of the evaluation criteria for each approach in the weighted normalized decision matrix from (d).The performance score is computed with the following formula.

Application of AMM and results
Business organizations have the goal of improving the maintenance of their business processes.Therefore, they need to measure the maintainability of business processes.Consequently, we will use the AMM approach to derive metrics for the maintainability of a business process.

Definition
The first phase is the definition of the goal, questions, and metrics.The goal is depicted with G, a question is depicted with Q, and a metric is depicted with M. In order to achieve the goal, we identify questions that need to be answered and the metrics for answering the questions.
G: Analyze a business process model to evaluate its maintainability from a model point of view.Q1 How easy is it to read or understand a model?M1.1 Control Flow Complexity (CFC), which is the metric for the complexity of a process model.Q2 How easy is it to modify an activity?
M2.1 The total duplicate occurrences of nodes, which is the metric for the redundancy of a process model.Q3 How easy is it to transform or change a model?M3.1 Minimal number of change operations needed to transform a process model to the desired variant model, which is the metric for the flexibility of a process model.
The interdependency of the attributes of a business process can be seen in Figure 4, which gives us more insight into how the complexity, redundancy and flexibility of a business process are related to maintainability.
Figure 4 Interdependency mapping of the attributes of a business process model

Data Collection
For the data collection phase, we manually measure the redundancy, complexity and flexibility (using the metric defined for these attributes in the first phase) of a corpus of business processes created in different business process modelling approaches.
Twelve business processes were used from existing literature with each business process having different variants.The twelve business processes were chosen by searching from existing literature and selecting the business processes we could get our hands on which have two or more variants.They are foreign exchange (FX) and money market (MM) process, construction plan process, and DNA sequencing process (Milani et al., 2016); vehicle repair process (Hallerbach et al., 2010); airline booking process in (Ayora et al., 2015); picture postproduction process in (La Rosa et al., 2017); invoice verification process (Rosemann and van der Aalst, 2007); equity trade process (Milani et al., 2012); loan application process (Buijs and Reijers, 2014); network diagnostic process (Lu et al., 2009); the student enrolment process model (Subic and Dimitrijevic, 2015); and a hypothetical business process (Pourmasoumi et al., 2015).
The business process modelling approaches we compared are the multi-model approach and single model approach (Marconi et al., 2009), the configurative consolidated model approaches which are the Configurable Event-Driven Process Chain (C-EPC) approach and Configurative Process Modelling approaches (Becker et al., 2004(Becker et al., , 2007;;Rosemann and van der Aalst, 2007), Process Family Engineering in Service-Oriented Applications (PESOA) (Puhlmann et al., 2005), Provop (Hallerbach et al., 2010;Reichert et al., 2009), proCollab (Mundbrod and Reichert, 2017).The details of the measurement of redundancy, complexity and number of minimal change operations for 12 businesses modelled in the different business process modelling approaches can be in the url 1 .Here, we describe how redundancy, complexity and flexibility are measured from business process models with the example of a picture postproduction business process modelled in different business process modelling approaches.The business process model in Figure 5a represents the "to-be" business process variant of the picture postproduction business process that will be derived from the picture postproduction business process ("as-is" business process model) modelled in different business process modelling approaches.We are using the picture postproduction business process presented in (La Rosa et al., 2017).

Multi-Model
Figure 5 shows the multi-model for business variants of the picture postproduction process, with each business process variant modelled separately using the BPMN modelling language.The variants Figure 5a, Figure 5b and Figure 5c each has a complexity of 0. The variant shown in Figure 5d has two ANDsplits with two outgoing edges, which results in a complexity of 2. The variant shown in Figure 5e has one XOR-split with two outgoing edges, which adds a complexity of 2, and the variant in Figure 5f has one XOR-split with two outgoing edges which adds a complexity of 2. The total complexity across the variants is 6 when the complexities for all variants are added together.The redundancy of all the variants is 27 as can be seen from Table VII.The minimal number of change operations is the number of activities added to create an entirely new variant model shown in Figure 5a, which is 5.The summary of the measurements are shown in Table VIII.

Single Model
The single model for the picture postproduction process is shown in Error!Reference source not found.Figure 6, and it is modelled using the BPMN standard.The model has a redundancy of 0. The model has four XOR-splits with two outgoing edges (CFC is 8), and two OR-split with two outgoing edges (CFC is 6), which makes the Control Flow Complexity (CFC) of all splits of the model 14.The minimal number of change operations needed to transform the single model to the "to-be" represented by the variant in Figure 5a is 6.In order to achieve the transformation, the following operations will need to be performed: deleting of "Prepare tape for editing", "Edit online", "Transfer in telecine", "Finish on tape", "Record digital film master", and "Release on new medium".The measurements for redundancy, complexity and the minimal number of change operations are summarized in Table IX.

C-EPC and Configurative Process Modelling
Figure 7 shows the EPC model for the picture postproduction process with the redundancy of functions equal to 0. The model has three OR-split with two outgoing edges each, which results in a Control Flow Complexity of 9.The minimal number of change operations needed to transform the C-EPC model in Figure 7 to a variant represented by the model in Figure 5a is 6.This transformation is achieved by deleting the activities: "Prepare tape for editing", and "Edit online"; and skipping: "Transfer in telecine", "Finish on tape", "Record digital film master", and "Release on new medium".The functions with tick outline such as the "Transfer in telecine" can be included or skipped depending on the result of the evaluation of variables.The measurements for complexity, redundancy and minimal change operations are summarized in Table X.

Provop
The main process model for the picture postproduction, which is based on the variant model shown in Figure 5d with related change operations is shown in Figure 8.The model has a redundancy of 0, and a complexity of 2, which is added by the two AND-split.The minimal number of change operations needed to transform the main process to the variant represented by the model in Figure 5a is 4 because the activities, "Prepare tape for editing", "Transfer in telecine", and "Finish on tape" will be deleted, and "Finish on film" will be moved as shown in Figure 8.The measurements are summarized in Table XI.

proCollab
Figure 9 shows a templet for the picture postproduction process which is based on the variant in Figure 5d, with a complexity of 2, because of the two AND-split and redundancy of 0 since there are no duplicate activities.To transform the model in Figure 9 to a proCollab variant templet which is equivalent to the variant in Figure 5a, the change operations, delete "Prepare tape for editing", "Transfer in telecine", and "Finish on tape"; and move "Finish on film" will be performed.Therefore, the minimal number of change operations for the transformation is equal to 4. The summary of the measurements for redundancy, complexity and minimal change operations are shown in Table XII.10 shows the PESOA model for the picture postproduction process with a redundancy of 2. It has one XOR-split with two outgoing edges (adds a CFC of 2), one OR-split with two outgoing edges (adds a CFC of 3) and one AND-split (adds a CFC of 1), making the complexity of the model to be 6.The minimal number of change operations for transforming the model in Figure 10 to the "to-be" variant represented by the model in Figure 5a is 6.In order to achieve this transformation, five subprocesses and one activity will need to be deleted.The measurements for the redundancy, complexity and the minimal number of change operations are summarized in Table XIII.

Results
The code for the business processes that will be used in the results is shown in Table XIV.

Conclusion
In this paper, we proposed an aggregate metric model (AMM) for evaluating business processes, which is an extension of the Goal-Question-Metric (GQM) approach for deriving metrics used for measuring business processes.AMM aggregates the different metrics of a business process which represents the different views of stakeholders by applying the Weighted Sum Method (WSM).AMM enables us to represent business process metrics with multiple attributes by aggregating the metrics of the attributes.AMM approach also describes how a corpus of business process models in different business process modelling approaches can be evaluated.Various approaches for measuring business processes are reviewed in this paper.However, these measurement approaches are not designed to use a multi-criteria metric for evaluation of corpus of business processes created in different business process modelling approaches, to make comparisons of such business process modelling approaches.
The evaluation of a corpus of business process models using AMM enabled us to gain insight into business process modelling approaches.For instance, from the result of the evaluation shown in Table XX we can see that business process modelling approaches which imbibed the principle of separation of concerns, make use of reference model and promote the reuse of model elements performed highly in terms of maintainability of business process models.The insight gained into the maintainability business process models with the applications of the AMM framework proves to be useful for choosing a business process modelling approach.AMM has enabled us to evaluate the trade-off between attributes of a business process model and has shown that the metrics of the business process can be integrated to give an insight into the goal an organization seeks from a business process.
A limitation in AMM is that we are not sure of how to reasonably assign weights to each evaluation criterion of a business process which is a constituent metric of the aggregate measure.Yen, (2009) suggested taking stakeholders' preference into account in assigning weights (Yen, 2009).

Figure 1
Figure 1 Mapping of the interdependency of business process attributesFigure1indicates that maintainability is the most important attributes of the business process.Attributes with more arrows pointing to them tend to be at a higher level of detail while attributes with fewer arrows pointing to them tend to be at a lower level of detail and more closely related to the structure of the business process model.Therefore, the attributes at a lower level of detail are more likely measured with metrics related to the structure of a business process model.

Figure
Figure 2 S1 process model

Figure 5
Figure 5 Multi-model for picture postproduction process (La Rosa et al., 2017)

Figure 6
Figure 6 Single model for picture postproduction process

Figure 7 C
Figure 7 C-EPC model for picture postproduction

Figure
Figure 8 Provop model for picture postproduction

Figure 9
Figure 9 proCollab model for picture postproduction

Figure
Figure 10 PESOA model for picture postproduction

Table I Order Matrix for S1
we start by drawing the truth table shown in Table V and then transfer the output of the truth table to a Karnaugh map.The Boolean expression is simplified with the Karnaugh map shown in Table VI.

Table XVIII A decision matrix for approaches and evaluation criteria
Normalization is performed to transform the metrics for redundancy, complexity and number of minimal change operations in the decision matrix in Table XVIII to a compatible unit scale, which results to the normalized decision matrix in TableXIX.The redundancy, complexity and the minimal number of change operations are cost criteria, and therefore the respective normalization formula for cost criteria presented in Section 3 is used.

Table XIX Normalized decision matrix for approaches and evaluation criteria
WSM is used to derive the weighted normalized decision matrix in Table XX from the normalized decision matrix in TableXIX.The overall score of redundancy, complexity, and the minimal number of change operations for each approach shown in Table XX represents the maintainability for that approach, as was described in the AMM framework in Section 3.