Applied Soft

We develop a novel method that uses single-valued neutrosophic sets (NSs) to handle independent multi-source uncertainty measures affecting the reliability of experts’ assessments in group multi-criteria decision-making (GMCDM) problems. NSs are characterized by three independent membership magnitudes (falsity, truth and indeterminacy) and can be employed to model situations characterized by complex uncertainty. In the proposed approach, the neutrosophic indicators are deﬁned to explicitly reﬂect DMs’ credibility (voting power), inconsistencies/errors inherent to the assessing process, and DMs’ conﬁdence in their own evaluation abilities. In contrast with most of the existing studies, single-valued NSs are used not only to formalize the uncertainty affecting DMs’ priorities, but also to aggregate them into group estimates without the need to deﬁne neutrosophic decision matrices or aggregation operators. Group estimates are synthesized into crisp evaluations through a two-step deneutrosophication process that converts (1) single-valued NSs in fuzzy sets (FSs) using the standard Euclidean metric and (2) FSs in representative crisp values using defuzziﬁcation. Theoretical and practical implications are discussed to highlight the ﬂexibility of the proposed approach. An illustrative example shows how taking into account the uncertainty inherent to the experts’ evaluations may deeply affect the results obtained in a standard fuzzy environment even when dealing with very simple ranking problems. © 2018 Elsevier B.V. All rights reserved.


Introduction
Multi-criteria decision-making (MCDM) is a collection of tools and methods used to solve problems with multiple and often conflicting criteria. Group decision-making is the process of making a decision based on feedback from more than one decision maker (DM). Group MCDM (GMCDM) is a complex process involving multiple criteria and multiple DMs. This complexity is amplified when the process involves qualitative and quantitative judgments on the potential alternatives with respect to the relevant criteria. These judgments are often vague and contradictory, and significantly complicate the construction of knowledge-based rules and the establishment of decision support procedures. Vagueness can occur under the following six circumstances: (1) the words that are used in antecedents and consequents of evaluation rules can mean different things to different people [1,2]; (2) consequences obtained by polling a group of experts are often different for the same rule or statement because the experts are not necessarily in agreement [3,4]; (3) decision groups are often heterogeneous due to the different extent of its members' expertise, knowledge and experience [5,6]; (4) expert estimates of criteria importance or performance of alternatives with respect to intangible parameters are not always consistent [7]; (5) information provided by individuals is usually incomplete or ill-defined [8,9]; and, (6) DMs are not always confident about the correctness of their own reasoning [10].
Fundamentally, uncertainty is an attribute of information [11]. There are two main types of uncertainties: external and internal. The external (or stochastic) uncertainty implies that the events or statements are well defined, but the state of the system or environmental conditions lying beyond the control of the DM might not be known completely. The internal uncertainty (or fuzziness) refers to the vagueness concerning the description of the semantic meaning https://doi.org/10.1016/j.asoc.2018.07.020 1568-4946/© 2018 Elsevier B.V. All rights reserved. of events, phenomena, or statements themselves, including uncertainties about DM preferences, imprecise judgments and ambiguity of information [12,13]. In this regard, Zadeh [14, p. 28] wrote: "As the complexity of a system increases, our ability to make precise and yet significant statements about its behavior diminishes until a threshold is reached when precision and significance (relevance) become almost mutually exclusive characteristics." Therefore, precise quantitative analysis is not likely to have much relevance in problems which involve humans either as individuals or in groups.
The presence of multiple vague measures in GMCDM has continued to challenge researchers and the problems associated with finding a comprehensive approach to modeling ambiguous information has still not been adequately resolved. In the sense of Ackoff [15], the problem of having ill-defined goals, ill-defined procedures or ill-defined data is a mess. Several theories have emerged during the last 50 years that generalize traditional probability theory and are more appropriate for not-probabilistic information formats in which evidence about uncertainty appears. These include Chiquet's theory of capacities, random set theory, evidence theory, possibility theory, Walley's theory of imprecise probabilities, fuzzy set (FS) theory, rough set theory, intuitionistic fuzzy set (IFS) theory and neutrosophic set (NS) theory, among others [16][17][18][19][20][21].
The most commonly used methodology for representing and manipulating imprecise and uncertain information in multi-criteria decision systems is the theory of FSs. However, while focusing on the membership grade (i.e., truthfulness or possibility) of vague parameters or events, FSs fail to consider falsity and indeterminacy magnitudes of measured responses. In practical terms, the problem of projecting multi-source and multivariate group decision uncertainty using mathematical models remains intractable in terms of FSs. In the late 90 s Atanassov [17] introduced and developed the idea of IFSs, intuitionistic logic and intuitionistic algebra allowing for more complex mental constructs and semantic uncertainties. In addition to the membership grade, IFSs consider non-membership levels. However, IFSs cannot handle all uncertainty cases, particularly paradoxes. NSs are the cutting-edge concept first introduced by Smarandache [20] in the late 90 s and developed in the 21 st century. NSs generalize FSs and IFSs. NSs and, in particular, singlevalued NSs are characterized by three independent membership magnitudes, namely, falsity, truth and indeterminacy. Such a formulation allows to model the most general cases of ambiguity, including paradoxes.

Contribution
This paper proposes a new approach to represent multi-source uncertainty of estimates provided by various domain experts in MCDM problems, and a methodology to integrate these measures within one decision support procedure.
Most of the existing studies on neutrosophic approaches to GMCDM problems focus on the development of aggregation operators to be applied to neutrosophic decision matrices in order to obtain group estimates of criteria and alternatives. At the same time, the truth, falsity and indeterminacy levels used to represent the uncertainty inherent to DMs' judgments are not given an explicit interpretation. These levels are usually treated as abstract triads of (non-standard) reals without highlighting their role as reliability measures or specifying the variables that they depend on.
The artificiality and routineness deriving from an overuse of aggregation operators together with the tendency to overlook a concrete interpretation for neutrosophic values within a given GMCDM problem represent a gap in the literature that need to be consider in order to investigate ways to effectively improve the applicability of decision-making processes.
The proposed GMCDM approach aims at increasing reliability, coherence and dependability of the final outcome by accounting for three different and independent reliability measures that can affect DMs' estimates, namely, DMs' credibility, inconsistency inherent to DMs' evaluation processes, and DMs' confidence in their own evaluation abilities. In order to do so, an assessment procedure for overall priorities of criteria and alternatives is developed using the technology of single-valued NSs.
More precisely, given a committee of M heterogeneous DMs/experts who must evaluate the performance of a set of I alternatives with respect to J criteria, we deal with the following problem. Problem: • Assumption: Let experts' estimates of the objects (I alternatives and J criteria) be affected by three diverse and independent factors: first, the experts have different credibility (i.e., voting power); second, the local priorities that are derived using relative comparison judgments are characterized by an inconsistency or an error measure; third, due to the lack of information or scarce experience, some experts do not feel confident about their own judgments. • Question: How can these uncertainty metrics be incorporated into a coherent ranking model to increase dependability of the group decision outcome? That is, how can multi-group multiperson expert judgments affected by these uncertainty metrics be coherently formalized and synthesized to yield reliable overall rankings of the criteria and alternatives?
Until recently, modeling and handling independent multisource uncertainties inherent to a single information unit was challenging due to the lack of appropriate formal tools. With the development of the NS and single-valued NS concepts, the problem of simultaneously handling different ambiguity indicators of one variable can be resolved by converting the values of the indicators into the truth-, falsity-and indeterminacy-membership grades of the corresponding variable. That is, the reliability of any estimate w provided by one of the expert m (i.e., the importance of a criterion or the performance of an alternative with respect to a criterion) can be expressed by a triad of independent magnitudes, ı m w , ε m w , Â m w , where ı m w represents the expert's credibility, ε m w the inconsistencies/errors intrinsic to the expert's evaluation process and Â m w the expert's confidence in his/her own ability and experience to evaluate the importance of the criteria and the performance of the alternatives.
After interpreting triads of reliability measures as neutrosophic values and group estimates as single-valued NSs, a deneutrosophication process is designed to synthesize crisp values representative of group priorities which are, in turn, used to estimate the overall performance of the alternatives.
It must be noted that the proposed formulation assumes independency among the alternatives' performances, i.e., synergy effects do not occur with respect to the alternatives' joint performance. Moreover, non-linear dependencies among criteria, in terms of their importance for the achievement of the overall problem objective, are not considered.
Finally, an illustrative example is provided to show how taking into account multi-source uncertainty indicators inherent to the experts' evaluations may deeply affect the results obtained in a standard fuzzy environment even in the case of very simple ranking problems.
The remainder of this paper is structured as follows. Section 2 offers a literature review focusing the recent applications of FSs, IFSs and NSs to GMCDM problems. Section 3 outlines the key features of the proposed NS-based GMCDM approach highlighting its differences and advantages with respect to the existing models as well as some theoretical and practical implications. Section 4 provides the necessary technical preliminaries and notations relative to FSs, IFSs, NSs, and the basic structure of a conventional GMCDM approach. Section 5 presents the aggregation and synthesizing mechanisms proposed for solving GMCDM problems with single-valued NSs. Section 6 presents an illustrative example to demonstrate the applicability of the proposed method. Finally, Section 7 draws the conclusion and some future research directions.

Applications of FSs/IFSs/NSs to GMCDM in the current literature
Complex problem solving is associated with the gathering of interest groups or experts to discuss the critical issues, such as for conflict resolution, for planning and design, for policy formation or for plan brainstorming [22,23]. Multi-criteria decision-making methods are an important set of tools for addressing challenging business decisions since they allow managers to better proceed in the face of uncertainty, complexity, and conflicting objectives [24]. Group multi-criteria decision support systems emerged in the late 80 s. They rely on the following four elementary stages [25]: (1) an initialization stage, where the general rules of the process to follow are determined; (2) a preference elicitation stage, where the individual DM expresses her/his estimates of the local weights of criteria and alternatives; (3) a group preference aggregation stage, where an analytical and synthesizing mechanism is used in order to derive a tentative collective decision; and, (4) a conflict-resolution stage, where an effort to reach consensus or at least attempt to reduce the amount of conflict is performed.
In most of the existing literature dealing with GMCDM problems under uncertainty, appropriate variants of FSs/IFSs/NSs are adopted to represent characteristics or performance estimates of alternatives with respect to a set of attributes or criteria. The variants used in the specific GMCDM problem depend on the level of uncertainty that is assumed to affect the DMs' evaluation process within the implemented assessment method.
Given their capacity of effectively represent the gradual changes of people's recognition to a concept in a certain context [26], FSs and IFSs have been widely used to describe intangible information. Usually, fuzzy or intuitionistic fuzzy numbers (trapezoidal or triangular) are used to interpret vague or imprecise information, with their linguistic variants employed to better deal with situations characterized by the presence of qualitative indexes. However, interval-valued FSs and IFSs are also widely used.
On the other hand, neutrosophic values (i.e., single-valued NSs, simplified NSs, interval-valued NSs, neutrosophic cubic sets, neutrosophic linguistic numbers) provide an effective tool to formalize independent degrees of indeterminacy, hesitation and/or uncertainty measures characterizing DMs' estimations in many real situations.
During the last two decades, the applications of fuzzy, intuitionistic fuzzy and neutrosophich values to GMCDM have increased exponentially, with dozens of researches and practitioners using them to represent DMs' uncertain preferences and construct aggregation rules based on fuzzy logic and/or neutrosophic inference.
In general, in fuzzy GMCDM environments, fuzzy or intuitionistic fuzzy values are used to formalize the uncertainty affecting the relative weights/priorities of criteria and/or alternatives provided by a group of experts. These values are used to define pairwise comparison decision matrices whose elements are manipulated to extract the DMs' crisp local weights/priorities. These priorities are synthesized in collective ones and, subsequently, aggregated to produce the overall preference values of the alternatives [27][28][29][30][31].
In NS-based GMCDM approaches, neutrosophic values are used to formalize DMs' estimates and construct neutrosophic decision matrices. These matrices are usually transformed in order to account for the attributes/criteria classification in benefit attributes/maximizing criteria and cost attributes/minimizing criteria. The transformed matrices are aggregated using aggregation operators specifically designed. A collective matrix is obtained and its elements are aggregated again to get the synthesized group priorities of criteria and alternatives. Finally, a score function (often combined with an accuracy function) is computed to rank the alternatives [39][40][41][42][43][44].
Furthermore, many models use the decision matrices to define an ideal alternative against which to compare all the given alternatives. One renowned way to perform these comparisons and identify the best alternative consists in calculating the (weighted) correlation coefficient between each alternative and the ideal alternative [53,54]. Other methods for comparing and ranking alternatives that have become common in FS-, IFS-and NS-based approaches to GMCDM are based on similarity measures [55][56][57][58][59][60][61][62] and cross-entropy measures [63][64][65][66][67][68].
In general, it must be underlined the key role that aggregation operators have played over the last two decades in solving GMCDM problems with intuitionistic fuzzy and neutrosophic information. However, following the pioneering work of Yager and Xu, who first introduced weighted arithmetic/geometric average operators [69,70] and power (average) operators [71,72], most studies have focused on developing new variants of generalized aggregation operators or improving existing ones instead of investigating ways to effectively increase the intuitive applicability of the entire decision-making process.

Key features of the proposed NS-based GMCDM approach: theoretical and practical implications
Our approach differs from those presented in the existing literature mainly in two ways.
i) Neutrosophic values are not used to construct the comparison matrices. The comparison matrices can be obtained through any crisp relative measurement method [73][74][75][76]. Neutrosophic values are introduced only after each DM has evaluated the local priorities to simultaneously handle three key different ambiguity indicators, namely, the credibility of the DM, the inconsistency characterizing/occurring in comparison judgments and the confidence of the DM in his/her own judgments. ii) The idea of ideal alternative against which to evaluate all the alternatives is replaced by the geometrical construction of an ideal neutrosophic reliability point, R * , against which it is possible to compare the neutrosophic values characterizing the local priorities of criteria and alternatives.
The theoretical and practical implications of the proposed procedure are strictly related to these two points.
Regarding the first point, the triads of truth, falsity and indeterminacy degrees to associate with the single DM' estimates are defined to explicitly reflect the DM' voting power range, the consistency ratios of the comparison matrices, and the DM' selfconfidence range, respectively. As a consequence, the collection of single-valued NSs constructed by collecting together all the experts' neutrosophic values relative to a given element (criterion or alternative) provides an intuitive but theoretically sound group performance estimates aggregation mechanism, without the need to define an aggregation operator. This is one of the advantages of the proposed procedure, namely, allowing for a natural aggregation of the DMs' judgments while concretely reflecting the three aforementioned reliability measures.
As for the second point, the comparisons with the ideal reliability point, R * , are purely geometrical: every neutrosophic value is identified with a three-dimensional point and its Euclidean distance from R * computed. The intuitive and operational simplicity that characterizes these comparisons constitutes another clear advantage of the proposed procedure.
Moreover, these comparisons lead to the definition of a deneutrosophication process capable of reducing the level of uncertainty of group estimates from neutrosophic to fuzzy, so as to allow (via defuzzification) for the crisp evaluations of global priorities.
Thus, the use of a geometrical ideal reliability point has the advantage of marking an easy-to-adjust link between the aggregation of uncertain local priorities and the syntheses of crisp global priorities. That is, it could be necessary to use mathematical objects more complex than single-valued NSs in the aggregation stage and/or modify part of the synthesizing (deneutrosophication) process. Being able of providing a suitable geometrical interpretation of the ideal reliability point is the key to adjust the proposed method to diverse situations and account for different modelling conditions.
The proposed aggregation and synthesizing mechanisms can be extended so as to allow for more general approaches requiring to formalize higher levels of uncertainty relative to DMs' estimates. For example, trapezoidal or triangular single-valued neutrosophic numbers (NNs) could be employed in place of single-valued NSs to represent DMs' priorities. This variant of the model would require some basic adaptations concerning the interpretation of priorities as trapezoidal or triangular single-valued NNs and the defuzzification method used in the synthesizing phase (see, also, Remark 1 in Section 5). Other relatively simple extensions could be obtained using interval-valued neutrosophic numbers.
One may also want to consider reliability measures for the experts' estimates different from those proposed in the current setting. For example, the truth-membership function could represent a measure of the quantity and quality of data/information available to the single DM instead of his/her credibility (i.e., voting power). At the same time, the indeterminacy of the estimates could be a reflection of the way DMs interpret the available data/information instead of being a measure of their self-confidence. In this case, DMs would be 100% confident in their evaluating abilities; the indeterminacy would follow from DMs' subjective ratings of the relevance, usefulness, completeness and precision of the available data based on their background and technical expertise.
From a more practical viewpoint, the flexibility of the proposed method can prove useful to managers and practitioners. For example, it can be applied to collaborative supplier selection problems with the purpose of incorporating both heterogeneous uncertain data and experts' subjective judgments within a hierarchical decision making structure. Similar applications can be developed in the medical and military sectors, among others, for assisting managers in the organization of human resources or the assessment of projects to be financed.

Technical preliminaries and notations
In this section, the foundations and differences among the most significant formal theories dealing with vague and imprecise information (primarily of the "internal" type) in humanistic decision-making systems are discussed. Afterwards, the conventional GMCDM approach to the problem of ranking a set of alternatives with respect to a set of selected evaluation criteria is outlined.

Basic notions of fuzzy sets
FS theory was first introduced by Zadeh in 1965 to formalize the gradedness in class membership, in connection with the representation of human knowledge ("linguistic" uncertainty) and uncertainty about facts [77]. Since then, FS have been relevant in the three types of information-driven tasks: decision-making problems, classification and data analysis, and approximate reasoning, for measuring degrees of similarity, preference and uncertainty [78]. FSs generalize ordinary (crisp, non-ambiguous) sets since elements can belong to them only partially according to the degree of "belonging" defined by a membership function. [11,18]). Let X be a universal space of points (objects) and x be a generic element of X. A FSẼ ⊂ X is characterized by a membership function Ẽ (x) : X → [0, 1], with the real value of Ẽ (x) at x representing the "grade of membership" of x inẼ.
A general continuous FS has the following representation: where M denotes the number of elements in X.

Property 1.
x ∈ X Ẽ (x)dx is non-negative and can be smaller or larger than unity. In the discrete case, 0 ≤ M m=1 Ẽ (x) ≤ M. Mathematically, this property illustrates the fundamental difference between the membership grade Ẽ (x) (sometimes also called possibility) and statistical probability p E (x) of the element or event

Property 2.
An element x ∈ X completely belongs (does not belong; partially belongs) to the FSẼ if and only if Ẽ (x) = 1 thenẼ is an ordinary (crisp) set. [11,18]). Defuzzification is a process mapping a FSẼ into a single crisp output e * ∈ 1 . That is, the set of pairs {(x| Ẽ (x)) : x ∈ X} is reduced to a single scalar quantity e * , which is a representative value ofẼ. The non-fuzzy (crisp) value e * best represents the possibility distribution of an inferred fuzzy action, so that certain concepts become clear, and certain goals and constraints are considered more relevant.

Definition 2. (Defuzzification
Van Leekwijck and Kerre [18] offer a survey of the existing defuzzification techniques classifying them into the three classes: maxima methods, distribution methods, and area methods. The most commonly applied defuzzification method is given by the center of gravity (COG): x ∈ X Ẽ (x) (for continuous sets)(for discrete sets) (1)

Basic notions of intuitionistic fuzzy sets
IFSs [17] extend FSs, as for each imprecise value, event or function along with a membership grade is defined a corresponding not-membership one. It is important to mention that both membership and not-membership grades are interconnected and that IFSs cannot handle indeterminate information. [17]). Let X be a fixed set. An IFS˙Ẽ ⊂ X is an object of the following form:

Definition 3. (IFS
the degree of membership and the degree of not-membership of the element x ∈ X, respectively.

Basic notions of neutrosophic sets
NS theory is a formal framework proposed by Smarandache in 1999. NSs are part of neutrosophy which studies the origin, nature, and scope of neutralities, as well as their interactions with different ideational spectra [20]. It generalizes the concepts of ordinary set, FS and IFS. In contrast to IFSs, in NSs, indeterminacy is quantified explicitly, while truth-, falsity-and indeterminacymembership functions are assumed independent. This assumption is very important in information fusion situations where it is necessary to combine data from different, possibly conflicting sources, e.g. sensors [79,80]. The set I N (x) may represent not only indeterminacy but also vagueness, uncertainty, imprecision, error, contradiction, undefined, unknown, incompleteness, redundancy, etc. [81,82]. Moreover, in order to better reflect vague information [79], the indeterminacy I N (x) can be split into subcomponents, such as "contradiction", "uncertainty", "unknown", and so on.  [80]). Let X be a universal space of points (objects) and x be a generic element of X. A single-valued NSÑ ⊂ X is a NS on X whose truth-, falsity-and indeterminacymembership functions are such that, ∀x ∈ X, TÑ (x), FÑ (x), IÑ (x) ∈ [0, 1].

Property 5.
Given a single-valued NSÑ, 0 < sup TÑ (x) + A single-valued NSÑ can be written asÑ = A unique feature of neutrosophy is that it can be used for modelling paradoxes. "The paradox is the only proposition true and false in the same time in the same world, and indeterminate as well" [10, p.13]. Definition 6. (Deneutrosophication). Deneutrosophication of a single-valued NSÑ is a process mappingÑ into a single crisp output Á * ∈ 1 . IfÑ is discrete, the vector of tetrads {(x|TÑ (x), FÑ (x), IÑ (x)) : x ∈ X} is reduced to a single scalar quantity Á * , which is a representative value ofÑ. The crisp value Á * is the one that best represents the aggregate distribution of the three membership measures TÑ (x), FÑ (x), IÑ (x) relative to an inferred neutrosophic element.

Conventional group multi-criteria decision-making approach
Let there be I discrete alternatives available for the selection, J intangible evaluation criteria, and M experts responsible for assigning importance weights to the criteria, as well as for estimating the performance of the alternatives with respect to the criteria. More precisely, let: Usually, a methodology that relies on a comparison principle, such as the Analytic Hierarchy Process (AHP) [7] or the Analytic Network Process (ANP) [75], is used by the experts in order to estimate the single alternatives with respect to each criterion.
Each is the priority of the i-th alternative with respect to the j-th criterion estimated by the m-th expert, and has dimension I × I.
To simplify notation, henceforth, B m will be used to denote any of these pair-wise comparison matrices while w m k (k = 1, ..., n) will stand for the priority of the k-th element (criterion or alternative) with respect to the higher-level element estimated by the m-th expert, with n = I for the set of alternatives and n = J for the set of criteria. That is: After applying one of the existing relative measurement methods, eigenvectors of ordered elements .., M} are the sets of derived experts' priorities of the k-th element. Moreover, each comparison matrix must undergo a consistency check to confirm transitivity and reciprocity of the judgment data. Homogeneity of the compared elements is another sufficient condition needed to assure good consistency [76]. For a detailed discussion of the relative measurement consistency issues, the interested reader is referred to [74,75,[83][84][85]. Following Saaty [73], the inconsistency measure of the comparison matrices B m is defined as the consistency ratio (C. In general, C.R. should be 10% or less. In some cases, the C.R. is required to be less than 8% (for n = 4) or 5% (for n = 3). Otherwise, the inference quality should be improved. The higher C.R. is, the less consistent and plausible the expert evaluations are.
The set of global priorities of the alternatives, W G , can be elicited by synthesizing all the individual priorities w m j relative to the j-th criterion and all the individual priorities w (j,m) i of the i-th alternative with respect to the j-th criterion. To accomplish this, the weighted geometric mean method can be applied (see, among others [86],). Once all synthesized values, namely, w j and w j i , ∀j = 1, ..., J, are revealed, the global priorities of the alternatives (w G i ∈ W G ) can be derived using the weighed-sum approach: The higher the weight w G i , the more preferable the alternative A i . For the best alternative, A * , it must be valid:

The proposed NS-based GMCDM approach
As highlighted in Subsection 1.1 and Section 3, we focus on defining a procedure that relates the single DM's priorities obtained through any methodology based on a comparison principle to his/her own levels of credibility, inconsistency and confidence. After being interpreted as single-valued NSs, the group levels are reduced to representative crisp values through an ad hoc deneutrosophication process. These values are, in turn, used to estimate the overall performance of the alternatives. Fig. 1 provides a graphical representation of the proposed NSbased GMCDM method comparing its stages with the typical ones of a crisp GMCDM problem and those generally implemented when solving GMCDM problems with fuzzy, intuitionistic fuzzy or neutrosophic information. The technical details of the proposed method are provided below.
Henceforth, the following additional notations will be adopted.

Defining single-valued NSs of group estimates
A discrete single-valued NSs can be defined for each set W k = {w m k : m = 1, ..., M}. This set contains all the experts' priorities derived for the k-th element: where ∀w m k ∈ W k , TÑ k (w m k ), FÑ k (w m k ), IÑ k (w m k ) ∈ [0, 1] 3 is the mapping of w m k into the neutrosophic space and: ε m k = 0 means that the m-th expert is fully consistent or unerring in her/his judgments regarding the relative weight of the k-th element. For priorities computation based on the right eigenvalue method employed in AHP, the judgments are fully consistent if and only if C.R.(B m ) = 0. In this case, the neutrosophic falsity-grade of the corresponding w m k is zero: FÑ k (w m k ) = 0. In AHP/ANP, an inconsistency exceeding 10% is not acceptable, therefore all priority vectors derived from inconsistent matrices are not relevant; that is, if C.R.(B m ) ≥ 0.1 then the falsity-grade of w m k is FÑ k (w m k ) = 1. If 0 < C.R.(B m ) < 0.1, then the corresponding estimates are still considered relevant to the decision-making process, but the closer the inconsistency is to 10%, the less credible the values w m k are (i.e., the falsity-grade of the priorities w m k grows). The neutrosophic reliabilities in Eq. (5) can be rewritten as follows: where VP(w m k ) (max(VP(w m k ))) is the (maximum possible) expert voting power for estimating the weight of the k-th element; ε k is the maximum acceptable inconsistency/error in the assessments of the priority of the k-th element; SC m k (max(SC m k )) is the (maximum possible) level of the m-th expert's self-confidence degree in estimating the priority of the k-th element.  reliability estimation in group decision-making. The general neutrosophic cube was first introduced by Dezert [87].

Single-valued neutrosophic cube and neutrosophic estimates reliability
The areas of reliability in Q are: unacceptable, high and tolerable.

Definition 7.
(Unacceptable neutrosophic estimates reliability). The area of unacceptable neutrosophic estimates reliability is represented by the three sides of Q : 1 = Q 2 Q 6 Q 7 Q 3 , 2 = Q 5 Q 6 Q 7 Q 8 and 3 = Q 3 Q 4 Q 8 Q 7 ; = 1 ∪ 2 ∪ 3 . In this area, the estimates w m k are characterized by 0% truth-, 100% falsity-and 100% indeterminacy-degrees, respectively. is a subset of W k that includes all the estimates w m k with zero truth-membership, unit falsity-membership or unit indeterminacy-membership: All the elements should be excluded from the decision process.

Definition 9. (Tolerable neutrosophic estimates reliability).
= Q ∩ ¬R ∩ ¬ is the area of tolerable neutrosophic estimates reliability. W k is a subset of W k that includes all the estimates w m k with below average truth-membership, above average falsity-membership or above average indeterminacy-membership: All the elements w m k ∈ W k have a minor impact on the group decision.
Once all single-valued NSsÑ k of group estimates are constructed, these values must be aggregated across M individuals to find a compromise priority of the k-th element for the committee. In terms of NSs, allÑ k must be deneutrosophied, and comparable/operable representative values Á * k ∈ 1 need to be elicited. The proposed deneutrosophication procedure includes two steps: first, conversion of all single-valued NSsÑ k into FSsẼ k , and second, defuzzification of the setsẼ k . ities w m k in the set are not reliable, they are assigned a zero-fuzzy membership grade. Hence: (10) In particular, for every w m k , w m k ∈ W k and m = 1, ..., M, means that w m k is more reliable than w m k .

Deneutrosophication of group estimates: step 2 of 2
FSsẼ k that represent uncertain group assessments with corresponding reliability grades are defuzzified using the COG method for the discrete case as defined by Eq. (1). Thus, one representative compromise crisp value e * k = Á * k ∈ 1 is obtained for each k: Normalization of representative opinions Á * k is needed to reflect their relative weights within the sets these opinions belong to:

Global priorities of alternatives
The overall priorities of the alternatives are derived using the weighted additive aggregation: where Á * j and Á * j i are the normalized representative values for the j-th criterion and the i-th alternative with respect to the j-th criterion, respectively. The higher is the value w G i , the better A i meets the collective group objectives.
Finally, note that the proposed method can be implemented using any common software such as Excel or MatLab. The numerical results of the illustrative example below were obtained in Excel.

Remark 1.
In the proposed method, trapezoidal or triangular single-valued NNs can be easily employed in place of single-valued NSs to represent DMs' priorities. This variant of the model requires some adjustments in the aggregation and synthesizing phases. We outlined below the changes to be applied in the triangular case. The trapezoidal case is similar. For the sake of completeness, recall that [88] a triangular single- valued NN,ã = (a, b, c); tã, fã, iã , is a NS on 1 with truth-, falsity-, and indeterminacy-membership functions defined as follows: . At this point, an appropriate defuzzification method must be implemented to defuzzify these numbers and synthesize the collective crisp priorities Á * k to be used in Eqs. (12) and (13).

An illustrative example
Let four domain experts with unequal voting powers be responsible for the assessment of three alternatives with respect to five criteria (M = 4, I = 3, J = 5). First, each expert builds pairwise comparison matrices as shown in Eq. (2) and applies the right eigenvalue method to reveal the weights of the criteria (w m j ) and the priorities of the alternatives with respect to each criterion (w with higher scores standing for greater DMs' influence. We assume that ∀i, j :, Moreover, the level of confidence in the correctness of the derived priorities is expressed by each committee member on a scale from 0 to 100, with lower scores designating greater indecision: ∀i, j, m, SC m j , SC Tables 1 and 2 presents the derived priorities of decision criteria and alternatives, the respective consistency ratios, as well as the confidence scores.  Table 2 Consistency ratio and confidence score estimates. Each set of group assessments w m j (respectively, w Also, ∀j = 1, ...J,Ñ j The single-valued NSs representing the uncertain group priorities of the alternatives are listed in Table 3. To fix the ideas, Fig. 3 provides the graphical illustration of the single-valued NSÑ C 1 A 1 that is, the space of group performance estimates of the alternative A 1 with respect to the criterion C 1 . Each element in the set {w (C 1 ,m)  -the estimates characterized by 100% falsity-and 100% indeterminacy degrees.
• The estimates with high neutrosophic reliability that imply a strong impact on the decision include: Table 3 Neutrosophic group estimates of alternatives with respect to criteria.
AlternativesA i Criteria C j Single-valued NSs of alternative estimatesÑ • The estimates with tolerable neutrosophic reliability that imply a weak to average impact on the decision are:  The calculations of the representative crisp estimates for each set of group members' uncertain values are enabled by using the centroid method and Eqs. (11) and (12). The relative compromise importance weights of the criteria are: Á * C 1 = 0.26, Á * C 2 = 0.195, Á * C 3 = 0.098, Á * C 4 = 0.248 and Á * C 5 = 0.199. The normalized representative values for the alternatives are given in Table 6. The global priorities of the alternatives are calculated using Eq. (13).
As shown in Table 6, alternative A 3 (Rank 1) has the highest global priority value and, therefore, it is the best one; alternatives A 1 and A 2 have close priority values, although A 1 (Rank 2) is slightly better than A 2 (Rank 3).   For a comparison, the global priorities of the alternatives under consideration for the same experts' estimates w m j and w = 0.3856 (Rank 1). It is evident from these comparative results that the uncertainty of the data is sufficient to affect the decision outcome.
For a more precise analysis, sensitivity of the results to changes in the experts' opinions can be tested. Special attention should be paid to the estimates that fall into the area of unacceptable neutrosophic reliability.

Conclusion and future research directions
We have proposed a novel method to handle multi-source uncertainty measures reflecting the reliability of experts' assessments in GMCDM problems based on single-valued NSs.
NSs are characterized by three independent membership magnitudes (falsity, truth and indeterminacy) and can be applied to model situations characterized by complex uncertainty, including those leading to paradoxical results.
Most of the studies on neutrosophic approaches to GMCDM problems that have been presented over the last two decades focus on the development of generalized aggregation operators to be applied to neutrosophic decision matrices in order to synthesize group estimates of criteria and alternatives. Furthermore, the truth, falsity and indeterminacy levels representing the uncertainty inherent to DMs' judgments are not usually given any explicit interpretation, nor are the variables that they depend on explicitly defined.
The artificiality and routineness deriving from an overuse of aggregation operators together with the tendency to overlook a concrete interpretation for neutrosophic values within a given GMCDM problem represent a gap in the literature that need to be consider in order to investigate ways to effectively improve the applicability of decision-making processes.
In the proposed approach, DMs' estimates have been characterized by independent tangible and intangible measures defined in a way to explicitly reflect DMs' voting powers, inconsistencies/errors inherent to the judgment process, and DMs' confidence in their own evaluation abilities.
At the same time, single-valued NSs have been used to formalize the uncertainty affecting DMs' priorities and aggregate them into group estimates without the need to define neutrosophic decision matrices or aggregation operators. The group estimates have been synthesized into crisp evaluations through a two-step deneutrosophication process that converts single-valued NSs in FSs using the standard Euclidean metric and, subsequently, FSs in representative crisp values through defuzzification.
An illustrative example has been provided to show how taking into account multi-source uncertainty indicators inherent to the experts' evaluations may deeply affect the results obtained in a standard fuzzy environment even in the case of simple ranking problems.
The proposed aggregation and synthesizing mechanisms can be extended so as to allow for more general approaches requiring to formalize higher levels of uncertainty relative to the DMs' estimates. In particular, a variant of the proposed method has been outlined where trapezoidal or triangular single-valued neutrosophic numbers (NNs) are employed in place of single-valued NSs to represent DMs' priorities.
Future research could focus on more complex extensions involving for instance interval-valued NSs, multi-valued NSs or linguistic variables. It could be also interesting to investigate reliability measures leading to algebraic definitions for the neutrosophic membership functions different from those proposed in the current setting.
From a practical viewpoint, the proposed method allows for applications to a wide range real-life situations. For example, it could be applied to collaborative supplier selection problems with the purpose of incorporating both heterogeneous uncertain/unknown data and experts' subjective judgments within a hierarchical decision making structure.
Similar applications could be developed in the medical and military sectors, among others, for assisting managers in assessing human resources or project proposals. Regarding, in particular, the medical sector, the proposed indeterminacy-membership function could be directly related to recent studies on judgment confidence and decision tendencies [89][90][91], where individual differences in decision-making are analyzed based on the idea that decision optimality is a function of judgment accuracy and tests are designed in the attempt to profile individuals' decision tendencies (i.e., optimal, realistic, incompetent, hesitant and congruent).