Obtaining Consensus from Mixed Groups: An Adapted Nominal Group Technique

out in collaboration between all Author HAH conceived the paper and wrote the first draft of the manuscript. Authors FLR, SW and MAD co-authored the paper. Authors HAH, FLR and MAD wrote the original study protocol for the Pharmacy study. All authors read and approved the final manuscript. ABSTRACT Aims: To describe an adapted Nominal Group Technique which can be applied to mixed populations. Study Design: Mixed methods Nominal Group Technique (NGT) consensus building exercise. of Study: Community Pharmacy across South West Wales, United Kingdom (UK) between 2009 and 2010. Methodology: We describe the research methodology involved in an adapted Nominal Group Technique. We carried out the adapted NGT exercise in homogenous consultation workshops following the traditional approach which resulted in the generation of individual consensus lists of the important issues related to the posed question. In order to ensure that issues highlighted within different consultation workshops were maintained in the outputs we further developed the NGT approach, firstly by bringing common issues together by applying a thematization process and then further collected during the research process, we further explored and justified the consensus reached. Results: We successfully applied the adapted NGT within the community pharmacy setting and were able to gain consensus regarding the positive and challenging aspects of patient- centred professionalism. Using qualitative data collected, we were able to describe in more detail the issues raised and the justification for the consensus reached. Conclusion: Using an adapted NGT, we describe how consensus was reached regarding the study question and how the basis for the consensus was explored. By applying this adapted NGT we were able to gain consensus regarding the relative importance of the issues under discussion across a mixed population group. The adapted approach allowed us to elaborate upon the consensus reached and justify the relative importance of the choices that were made.


INTRODUCTION
Consensus methods provide a means of synthesizing information, and where published information is inadequate or non-existent, a means of gaining insights on views or approaches from appropriate experts and stakeholders [1,2]. Consensus methods aim to determine the extent to which experts or lay people agree about a particular issue [1]. They also aim to overcome some of the problems associated with group decision-making processes, where dominant views may lead and crowd out other perspectives [1].
There are three main consensus approaches that have been used within the health field [3]. The earliest of these was the Delphi method developed by the RAND corporation [4,5]. This was followed by the Nominal group technique (NGT) [6,7] and the Consensus development conference [2]. There have been variations to these approaches, with one of the most notable being a modified NGT developed by the RAND corporation [8].
The NGT has been applied to numerous healthcare fields including the development of clinical guidelines, strategies for implementing new nursing practices, development of quality of life measures, appropriateness of out-of-hours working for junior doctors and priorities for health promotion [3]. Most commonly the NGT is applied to homogenous groups. In this paper we discuss the use of an adapted NGT where consensus regarding the relative importance of the issues under discussion can be obtained across mixed population groups. The approach allows the views of different individual groups to be maintained throughout the consensus process and not be diluted when combining data from multiple population groups. The adapted NGT approach also expands on the quantitative consensus results by embellishment with qualitative data collected during the research process. This enables more in-depth exploration of the consensus results and allows elaboration and justification for decisions regarding the relative importance of the choices that are made.

Nominal Group Technique (NGT)
NGT was first developed by Delbecq et al. in the 1970s as an organizational planning technique [6,7]. The process of NGT normally involves four main phases. Phase one is the nominal or silent phase, during which individuals consider their personal responses to a defined question and write them down without discussion. Phase two is the item generation phase, during which individual participants take turns to share their responses (related to the pre-specified research question(s)) with the wider group. Items generated are recorded as succinct phrases without being discussed within the group. Phase three involves a discussion and clarification phase, in which group members are allowed to ask questions in order to clarify items on the list and elaborate on their merits and demerits. During this process, items with similar meanings may be combined and duplicate items removed. The final phase is the voting phase, where each individual is asked to prioritize the listed items by assigning ranks to them. These individualized ranks are then collated to produce a single ranked consensus list for the group. NGT encourages individual contributions, and the inhibitory factors of group conformity are avoided with even the more reticent group member facilitated to participate in all phases [9].

Adapted Nominal Group Technique
We describe in this paper the methodological process involved in our adapted NGT whereby we elaborated upon the four phases above. As consensus techniques are more commonly applied to homogenous groups, this approach allowed us to consider multiple viewpoints and was therefore suitable for gaining consensus opinion from mixed groups of individuals, and allowed a full and in-depth representation of these views from all individuals involved. We recommend that the data collection processes for our adapted NGT take place in a number of phases:

Consultation workshops
The first part of the consensus exercise is undertaken across a series of consultation workshops. Separate workshops are organized for each of the relevant homogenous groups involved (for example, pharmacists, stakeholders, pharmacy staff and the public). Consultation workshops are facilitated group meetings taking place over an extended period of time to enhance inter-group dynamics [9]. Historically they have been used within evaluation research [6], for educational purposes [10] and group training [11]. Recruitment to consultation workshops is aimed at an optimal number of six study participants per workshop, in keeping with the literature on group consultation activities [12].
The consultation workshops begin with an exercise that encourages group discussion according to a semi-structured interview schedule. The schedule is pre-designed prior to the workshop and poses a number of open questions that are related to the specific issue under consideration. The design of the schedule is based on existing literature in the field and covers pertinent issues related to the matter on which consensus needs to be reached, but also has the flexibility to examine how these relate to personal experience.
Following the general semi-structured discussion phase, the specific research question to be addressed and on which consensus is sought, is posed to the homogenous groups within each of the consultation workshops. The generation of the item lists using the NGT then follows according to the standard approach outlined in the literature [1,9].
Median ranks with interquartile ranges (IQR) are calculated for each of the item lists generated by each separate workshops, following which a consensus ranked list is produced for each group based on these final median ranks (Output 1, Fig. 1).

Generation of themes
The second phase in the adapted NGT involves the generation of common themes from the outputs produced from each of the workshops. A complete list of items produced from all the individual consultation workshops is collated. Two members of the research team then independently combine the individual workshop items into common themes. This process follows the standard qualitative process of thematic assimilation based on linking common data to develop overarching categories or themes [13]. Each individual researcher organizes the items into a series of over-arching themes under which the items fit (Step 8, Output 2, Fig. 1). Where the posed question serves to identify positive and challenging aspects of a particular issue, these are allocated to the same over-arching theme (with the positive and challenging aspects stated clearly). The generation of themes based on the workshop findings allows all the items from the individual group workshops to continue to be in evidence.
In order to minimize researcher bias and to ensure that the generation of themes is as rigorous as possible, independent analysis of the item lists is conducted, with a least two individuals reviewing the complete set of items. The review of the item lists seeks to identify themes that are thought to encompass the posed question. No data are removed at this phase apart from deletion of duplicate items. Items that are duplicated across lists are amalgamated into a single item.
As a quality control step, a third individual independently checks the two generated thematic lists and amalgamates them into a one set of common themes that reflect the two individual thematic lists. This stage involves further condensing of the items encompassed within each theme by bringing together similar items and producing a new overarching statement, if necessary. By using a thematic process that only employs reduction, where duplication is clearly evident across several groups, the representation of important items within each group classification can be maintained.
The research team (at least 4 individuals involved with the whole research process) agrees the final thematic list to ensure all the items are included and that the themes generated adequately describe the items contained within them.

Thematic ranking
Following the thematic process, individual themed cards are produced by the research team based on the final agreed thematic list. A separate card is produced for each theme generated, with the theme subject as a header under which the associated items are listed.
All participants who attend the original consultation workshops are presented with the thematic cards and asked to rank them in order of importance: with '1' representing the theme they regard as the most important and subsequent ranks signifying those of diminishing importance. Participants are advised to rank only those themes they personally regard as important and to avoid joint ranks. Space is also provided for participants to supply qualitative feedback, if they so wish (Step 9, Fig. 1).
Data from the returned thematic cards are collated and median ranks and interquartile ranges (IQR) are calculated for each of the themes. A consensus ranked thematic list is then produced based on these median ranks. This list forms the basis for the discussion and generation of the final consensus list at a forum event.

Forum event
With the adapted NGT, a forum event is included in the methodological procedure to enable the production of a final ranked consensus list of themes that encompasses the items of the posed question(s), and that serves to clarify what these themes and items mean to mixed groups of participants. All the original consultation workshop attendees are invited to attend the forum, but at this stage of the process groups are purposefully mixed to enable participants to consider other participants' views. The forum is organized in a central location at a time which allows the greatest attendance.
In order to successfully facilitate discussion, attendees are divided into groups of a manageable size (6-10). One member of the research team facilitates each group, supported by at least one other individual for the purpose of consistency with other aspects of the methods.
The forum event begins with the presentation of the pre-forum thematic list (Step 10, Fig. 1). Participants are then asked to discuss in their mixed groups whether they regard the ranking to be an appropriate reflection of the relative importance of each theme (as defined by participants during consultation workshops) (Steps 10 and 11, Fig. 1). Following group discussion, participants are asked one final time to privately re-rank the themes. Data from the thematic cards are collated and median ranks and interquartile ranges (IQR) are calculated for each of the themes. The final mixed-group thematic list can is then generated (Step 12 leading to Output 3, Fig. 1).

Data transcription and template generation
During the individual consultation workshops and forum event, notes and audio recordings are taken and transcribed. Following the generation of the final thematic lists, the individual transcripts undergo thematic and summative analysis on both an individual and group level to extract detailed information relating to each of the generated themes [14,15]. The detailed content is built up under each of the themes to add depth of understanding and to clarify any anomalies regarding the important aspects of the posed question(s) [16]. (Output 4, Fig. 1).

Example: Examining Patient Centred Professionalism in Community Pharmacy
We successfully used the adapted NGT in a recent evaluation of the positive and challenging aspects of patient-centred professionalism within community pharmacy. We describe the detailed methodology only here. The details of the study findings are presented elsewhere [17,18]. When we used the adapted NGT within the context of this mixedmethods study, the quantitatively ranked consensus material highlighted the views of all participants, whilst allowing contextualisation alongside more in-depth qualitative findings.

Consultation workshops
In the example we refer to above, we recruited participants to seven individual consultation workshops: (1 public, 1 stakeholder, 2 qualified pharmacists, 1 pharmacy staff and 2 newly qualified pharmacists). We began the consultation workshops by presenting bio-photographic material from previous research studies within general practice and community pharmacy practice [19,20], with the purpose of being an 'ice-breaker' which encouraged a general discussion from the workshop participants. Questions such as "who works in these environments?, What do you think about the appearance of the environment? and How does this compare with where you work?, for example were presented to the workshop participants. We followed the general discussion with a more structured discussion using a pre-defined semi-structured interview guide. We identified pertinent issues from earlier research studies within the same field and from existing literature within the area of patientcentred professionalism. The aim of the semi-structured discussion was to encourage workshop participants to consider their own practice in relation to other professional perspectives and was followed by an extensive discussion, using personal, storied examples of experiences of community pharmacy to explore the nature and importance of patientcentred professionalism within people's public and professional lives. In the final element of the consultation workshops we carried out a NGT exercise, in which participants were asked to address the question: "what are the positive and challenging aspects of patient-centred professionalism within community pharmacy?" Through the NGT exercise, we produced two lists which contained approximately 10 positive and 10 challenging items relating to patientcentred professionalism for each workshop.
Once we had generated the list of positive and challenging aspects of patient-centred professionalism, we asked each workshop participant to rank the lists of items on a preprepared proforma. We also asked participants to add a qualifying statement alongside each of the ranks, if they so wished. We produced two ranked lists of positive and challenging items for each of the seven consultation groups.

Generation of themes
Following the completion of the seven individual workshops we grouped the complete list of positive and challenging items into a series of broad themes using a qualitative thematic approach. Two of the study team (FR and HH) independently reviewed the complete set of items (n = 54 positive and n = 53 challenging), in order to identify common themes that encompassed both the positive and the challenging aspects of patient-centred professionalism. SW then independently quality assured the two generated lists to produce one set of eleven common themes (safety, professional characteristics, relationships with patients, confidentiality and privacy, accessibility, training, professional pressures, services, environment, changing professional roles, and patient characteristics) that reflected and amalgamated the thematic processes of FR and HH. All members of the study team agreed the final thematic lists.

Thematic ranking
Following the thematic process, the research team produced a series of thematic cards. We placed the title of the broad theme at the top of the card and listed the associated set of positive (on the left) and challenging (on the right) items under that theme (Fig. 2).
We sent all participants who attended the original consultation workshops the thematic cards and asked them to rank them in order of importance. We collated the data received and calculated the median ranks and interquartile ranges (IQR) for each of the themes. Following analysis we produced a consensus ranked thematic list.

Positive Challenging
 Correct prescriptions for patients  Good hygiene  Adherence to codes of conduct and accuracy of service in the interests of patient safety and welfare  Increased workloads and pressures compromise on quality of service to patients  Mistakes in administering/delivery of medicines RANK Any comments:

Forum event
We organized the participants who were able to attend the forum into mixed groups. We began the forum event by presenting the pre-forum consensus ranked thematic list. We then posed the question "Does the ranked thematic list reflect your view regarding the important positive and challenging aspects of patient-centred professionism?" Mixed group discussion then followed, after which we asked each participant to privately re-rank the themes one final time. We collated the final responses and calculated the median and interquartile range for the ranked themes. We produced a final thematic list which was based on the mixed groups' views [17,18]. For the posed question "what are the positive and challenging aspects of patient-centred professionalism within community pharmacy?" we produced the consensus list of (in ranked order): safety, professional characteristics, relationships with patients, confidentiality and privacy, accessibility, training, professional pressures, services, environment, changing professional roles, and patient characteristics [17,18].

Data transcription and template generation
Each member of the research team was allocated themes identified by the adapted NGT process. Each research team member then carried out thematic [14] and summative [15] analysis of each transcript from the 7 consultation workshops and the forum event. This ensured that we extracted all the relevant information relating to each of the themes. We independently scrutinized and cross-referenced the extracted information with the original transcripts to ensure data reliability and validity. We used the qualitative data extracted to build up a coherent and in-depth picture of the underlying meaning of the generated themes and the items contained within them. Fig. 3 illustrates a brief summary of one of the themes generated from the study. Although our adapted NGT follows the same initial processes defined by the original authors [6,7,9], we have modified it into a more staged process involving firstly individual consultation workshops, and then more latterly a mixed consultation process. Using the elaborated process of in-depth thematization of the ranked consensus lists alongside the final mixed forum event of this adapted NGT we were able to provide a mechanism for gleaning an understanding regarding the consensus opinion from a mixed group of patients and professionals. Traditionally, the NGT is applied to homogenous groups of individuals. Our adapted approach therefore allows the views from multiple mixed population groups to be considered and assimilated. One of the main advantages of employing a consensus technique is that the more reserved participants of the workshops or focus groups all have the opportunity to contribute their ideas [10]. Although consensus techniques have drawbacks, particularly related to whether the outputs are representative of the broader population, they are increasingly being applied to resolve or define key issues within the healthcare arena. Indeed, there have been recent recommendations to employ such methodological approaches to address unanswered questions [21] as well as to carry out further research to improve consensus development methods [4].
By employing the processes of thematization and qualitative elaboration to the NGT we were able to consider and represent the views of all the participant groups in our case study. In addition, the final forum event facilitated the engagement and further contemplation of different participant views.
It could be suggested that the design of the adapted NGT may result in biased results due to participant drop-outs (commitment to attend on a number of occasions, geography may limit attendance etc). It has been contested by qualitative researchers that bias is problematic in

Theme 1. Safety
Safety is widely regarded as the most important aspect of community pharmacy, so much so, that for many it almost goes without saying. Working practices, working relationships and workspaces are all configured to sustain and support patient safety yet at the heart of patient safety is not the patient but medicines. Consequently, many pharmacists are torn between communicating effectively with patients and safely dispensing medicines. This tension is exacerbated by: the growing emphasis on patient engagement, the creation of new roles and responsibilities, heightened commercial expectation especially around performance, and an intensification of the culture of responsibility and accountability, with the bureaucracy that this requires. Individual pharmacists feel the strain of baring the risk of dispensing and checking errors, leaving them in a constant state of anxiety. They are frustrated that the public appear neither aware of their dedication nor appreciative of the fact that they work so hard to ensure patient safety. However, the imperative to work safely is also a source of identity and pride. The public, for their part, are acutely aware of the implications of dispensing errors and the seriousness of the pharmacist's job, and many suggest pharmacies should be limited to objects: dispensing not diagnosis.
health research [22]. Although the research design may indeed result in drop-outs, as the adapted NGT supplements the consensus ranking process with qualitative data obtained at various stages in the process, this ensures that views from participants who attended for only part of the process are still in evidence. The ranking process is a way of distilling and presenting the summarized findings and we recognize that the relative importance of these ranks may be different depending on the size and make-up of the groups involved and the context and the specific relationship between the individual groups involved.

CONCLUSION
The adapted NGT provided a mechanism for distilling the important components of the posed question in a mixed group of individuals. The qualitative elaboration process provides insight into what consensus ranked thematic lists mean to all the individuals involved. This process also allows the development of a more comprehensive appreciation and understanding of the relative importance of the themes generated.

CONSENT
All authors declare that all participants in the Pharmacy study were provided with written information prior to taking part and signed a consent form indicating their willingness to participate.

ETHICAL APPROVAL
Based on Section 2.3-Governance arrangements for Research Ethics Committees (GafREC) published by UK Health Department, May 2011, due to its situation within Community Pharmacy, the South West Wales Research Ethics Committee, UK deemed that the case study was classified as a service evaluation, thus not requiring ethical approval.