Co-design of a crowdsourcing solution for disaster risk reduction

Disaster Risk Reduction (DRR) is a complex field in which a huge amount of data is used to plan preventive measures, get prepared to natural disasters, and effectively respond when they strike. This work focuses on the definition of a co-design methodology to integrate a crowdsourcing solution in the DRR processes. We define the proposed methodology, and implement it involving operators and experts in the DRR domain (crisis managers, technical services, first responders). We show how a participatory design approach helps in the design of a crowdsourcing solution that experts are willing to integrate into their DRR procedures.

enter as fundamental bricks of the disaster management process [2]. This perspective requires institutions to include in their procedures the data collection process, which could be performed also from informal and public platforms [3], even if the information management during disasters is centralized and strongly hierarchical. Considering the actors involved in all phases of the DRR ecosystem ( Figure 1), it is easy to understand the value that socio-technical paradigms and novel systems can offer. ICT and in particular mobile technologies are enabling a paradigm shift, supporting active participation, resources sharing, spontaneous feedback and distributed sensing. All these activities are already adopted by citizens in everyday life and they could be beneficial also in the DRR cycle to overcome the lack of information before, during and after a disaster [1,4]. Technology replies to the natural attitude towards active contribution that people show if directly and indirectly involved in the disaster management, where the more frequent needs are related to real-time geo-localized information from the field, effective communication of warnings and advices, requests of help and assistance [5]. The collection and sharing of geospatial information from people with reference to a common context via Internet-enabled mobile devices is also referred as crowdsourcing [6]. It can be adopted to shape cooperative processes conveying the potential offered by self-organized information generation and sharing, delivering actionable data for crisis managers. Thanks to mobile devices and networks, citizens can become digital volunteers [7], self-engaged in environmental monitoring and caring activities. The data coming from local residents, reporters, first responders or from authorized information sources contribute to better manage emergency situations. This is a particularly hard challenge due to the complexity of the context, the variety of the involved actors, with different responsibilities and needs, as well as the technology constraints related to the communication channels availability and reliability. In this work we tackle this challenge by proposing a methodology that enables the definition of a crowdsourcing solution for DRR, and implement it involving several actors and top-notch experts in disaster management through a participatory approach. The rest of this paper is organized as follows. In Section 2, we review related works, focusing on the crowdsourcing approach and the on the co-design framework. In Section 3 and Section 4, we detail the methodology and its implementation, respectively. Finally, we draw results in Section 5, while in Section 6 we outline conclusions and future works.

RELATED WORKS
According to the literature, the most recent disaster management approaches include in different stages the direct participation of citizens [4,8]. In particular, in critical conditions, regardless of the emergency type, people need to actively seek and share information, while staying in contact. Mobile technologies allow to fulfil this need [9], and enable the crowdsourcing. This socio-technical paradigm leverages on the willingness of people to participate in processes which benefit the community [10]; and benefits from current mobile technologies, which are used as sensors to gather and exchange geo-located information (text, videos and images) about a crisis [11,12]. Crowdsourcing typically involves large numbers of cooperating people. The larger the voluntary contribution in sensing is, the more consistent the data collection in real-time from the field will be. The large amount of useful information would be a precious resource for first responders, non-profit humanitarian organizations, local and national governments [13]. The main advantage that makes the crowdsourcing interesting is the capability to bring the local knowledge from the territory and citizens to the decision makers. Nevertheless, it exhibits some critical aspects. One of the most sensitive point consists in the reliability of the produced information, which can suffer from disinformation, false judgments, social biases [14,15]. In addition, the skill level of the contributors affects the accuracy of the collected data and their further use. This is especially important in the fields of surveillance and crisis management [16]. Automatic data filtering techniques based on machine learning techniques [17] are advisable in order to cope with the potential high volume, velocity, and variety of the data being collected, and help decision maker to focus on actionable data while excluding not informative or misleading contents. Validation scheme range from automatic systems to the human verification, including the integration of articulated reputation systems that dynamically rate the contributors [18]. Finally, the outputs of the crowdsourcing, require proper visualization technique as well as technological equipment. All these issues have to be considered when designing and introducing crowdsourcing approaches in safety-critical processes like DRR. For these reasons, the design strategy can profit of user-centred methods and techniques. The user centricity in the design process [19][20][21] aims at collecting and understanding end-users' needs and the context of activities. The co-design methodology focuses on a collaborative path according to the claim "Building With, Not For" [22], involving in a collaborative process users and other domain experts as equal partners as designers. Co-design sessions are based on working techniques stimulating the dialogue on a common problem and possible solutions. In practice, the co-design allows non-designers to articulate design proposals as a useful starting point for further professional developments [23]. The expected output of co-design activities is a set of assets containing information, suggestions, constraints, to be further elaborated and then conveyed into a concept that dynamically evolves from the early stage to the final User Experience (UX), defined as a set of "perceptions of the users and results of the use and/or expectation of use of a product, system or service" [24].

A METHODOLOGY TO CO-DESIGN A CROWDSOURCING SOLUTION FOR DRR
How a crowdsourcing solution can enable the collection of relevant, reliable, and actionable data, to be integrated into current DRR processes so as to be useful to crisis managers and operators? We address this question proposing a methodology based on a co-design approach, structuring it in four main phases (see Figure 2). The methodology can be applied to single individual or groups. In the following we assume the latter condition.
Phase1: Introduction and initial needs. At the beginning, participants receive a brief introduction to the goals of the activity, the process, and the timing. Subsequently, participant introduce themselves to the plenary audience, highlighting their skills, interests, the most impelling needs of their organization in the field of the DRR ( Figure 2, box 1). To harmonize presentations, a common template should be provided before the gathering.
Phase 2: Data Scouting. This phase aims at eliciting and sharing the information useful in the current DRR cycle, for different types of hazards. Groups are asked to freely list [25] the in-field information needed by DRR managers sitting at the control room, with no worry on data properties such as source, format, acquisition media, accuracy, etc. Then, they are asked to classify the collected information by hazard, DRR phase, importance, and data provider (citizen or professional).
Phase 3: Prototyping. In this phase, groups are called to put in practice the outcomes of the previous phase by doing a prototype aimed at transforming the emerged informative needs and requirements into a conceptual model of the system. One or more specific scenarios are given to each group, asking them to create a user interface for in-field users containing the information they would like to provide or see in the considered scenario. Each scenario refers to a specific hazard, a DRR phase, and data provider. Applied in cooperative environments, this technique (usually referred as co-sketching by UX professionals) allows each group to consider alternatives and to integrate different ideas and solutions. It is a quick and effective way to keep the process modular and collaborative. It leads the participants to ask themselves critical questions about the actions the user needs to take, and consequently to elicit the information he/she needs to take those actions. The resulting wireframes are low-fidelity visual representations of what the system should do, communicate, and of how it will look like to be coherent with the reference target [26]. At the end of the prototyping phase, all the artefacts produced in phase 3 are publicly exposed, and each artefact is explained to the audience by the group leader (autonomously elected) to facilitate discussions, and dive more into the reasons behind the design choices. In this way, all participants are invited to discuss divergences, share priorities, obstacles, ideas and solutions. We recommend to deserve enough time for the groups' presentations and plenary discussion to consolidate common understandings and to reach consensus on the most effective ideas and solutions [27].

IMPLEMENTATION
We implement the proposed co-design methodology within the framework of the I-REACT project 1 , where the contribution of the crowdsourcing is twofold: (1) to empower emergency operators with real-time information gathered from the field and (2) to engage citizens as additional data provider. As a preliminary step, we perform a survey aimed to collect a first knowledge about our endusers, and we analyze its outcomes to organize the agenda of the co-design workshop, the scenarios of the prototyping, the hazards to be considered, and design dedicated toolkits. Then, we conduct the co-design workshop following the 4 phases of the described methodology. Finally, we analyze all the gathered outcomes and draw conclusions. In the following we explain in details each of the aforementioned implementation steps and results.

The survey
Each I-REACT consortium member were invited to formulate closed questions and reply options to the project stakeholders (166 questions in total). Such questions relate to the most impacting hazard to be addressed, the most useful DRR practices and tools, crowdsourcing opportunities, the use of social media and advanced cybertechnologies in the different phases of the DRR cycle. We organized all collected questions by topic, creating an online form that we share via e-mail with the end-user panel. Several profiles from 11 different organizations in Europe replied to the questionnaire. In particular: crisis managers (46%), policymakers (18%), technical experts (18%) and in-field agents (9%). By analyzing the replies, we mapped an early set of requirements and build a preliminary knowledge. The most relevant result concerned the positive attitude 1 www.i-react.eu toward the possibility to receive data from the field by means of a crowdsourcing approach. According to the involved organizations, the Preparedness, followed by the Response are the DRR phases that would benefit more by the introduction of a crowdsourcing solution. The volunteer resulted the role that would best facilitate the crowdsourcing introduction in DRR processes. Thanks to their training, volunteers are a reliable source of information from the field, both for detecting hazard specific measures (such as water level, water speed, water volume) and for weather measures (such as air temperature, wind speed, rain, hail). On the other side, the panel appears much more undecided concerning the importance of involving "common citizens" in this kind of quantitative reporting.

The co-design
General Organization. We organized an international two-day workshop, during which emergency responders and carefully selected international advisors interplayed with I-REACT system developers. We involved 52 participants: 23 stakeholders (11 emergency organizations and 12 advisors), 29 members from the project consortium, including 8 facilitators. Before starting with the first phase, we presented an overview of the I-REACT project to give a common context to all participants. After that, we asked every stakeholder to give a 3-minutes (3 slides) presentation following a pre-defined format we provided before the workshop. The presentation included key information: main activities, role, competences, and the main needs in the DRR. We randomly divided the participants into 5 groups of average size of 9 members, balancing the number of project members and stakeholders. We collect the output of each phase by electing a consortium member within each group as scriber to note down the key discussions, including both divergences and agreements. We also took pictures of the artefacts at each intermediate phase, and recorded the collective sessions. Additionally, during breaks we interviewed participants in order to get their feedbacks in real-time and fine tune the process accordingly. We devote 1 hour and a half for each phase after the introduction, namely the data scouting, the prototyping and the discussion. After the workshop, we devoted two weeks to analyze all the collected material (videos, photos, sketches, notes) and draw the main outcomes, which are highlighted next.
Data scouting. We divide the data scouting into two phases. First, we let groups to freely list the information required from the field and map them to hazards, next we asked them to assign each data a priority, the involved DRR phases, and decide whether the data could also be provided by citizens. We distribute to each group a Venn diagram with three overlapping circles, one for every of the three identified hazards (flood, fires, weather extremes), and ask participants to stick post-it on top of the right diagram portion, one per data to be collected from the field. A matrix was used to let groups assign to each information a priority flag, the DRR phases involved, and whether the content could be provided also by citizens. (Figure 2, box 2) Prototyping. In order to be more productive, for the sketching phase we divided each group into two equally sized subgroups. Hence, we created 10 smaller groups keeping at least one end-user, one expert, and one consortium member inside each group. We assign to each group a specific scenario, where the scenario is characterized by the hazard, the user role (professional, citizen) and the phase of the DRR cycle. The use of micro-scenarios allows to better frame the context, hence improving the specificity and the quality of the final output. As before, in this phase all groups work in parallel, but the task assigned is to co-sketch a paper mockup, keeping in consideration the outcomes of the previous phases. We randomly assigned scenarios to each group and we gave as supporting tool (i) paper wireframing because it allows for flexibility ( Figure 2, box 3), (ii) stencils (equal for all groups) containing the most common user interface patterns to be cut and pasted, modified, colored, and combined on the screen template. We choose to provide a mobile screen template, allowing a single screen to be filled so as to force the group to carefully select the most meaningful data and functionalities and, by drawing it, to visually organize them.

RESULTS
Current needs from the participants' introduction. The results confirm the heterogeneity of the stakeholders group. In line with the project focus, the most important hazard is flood, followed by weather extremes. The use of maps and European mapping services such as Copernicus EMS 2 frequently appeared among the used tools and open data. Real-time reporting is confirmed to be the most relevant requirement for end-users, while dynamic risk maps are 2 http://emergency.copernicus.eu/mapping  ranked second. The latter refer to the capability of risk maps to be recomputed very frequently according to the changing conditions (hazard, exposure, vulnerability). Greater anticipation of hazards in the early warning phase and synthesis of content from social media rank third and fourth, respectively. Advisors highlighted the importance to integrate information from different sources such as satellites and sensors. They seem to consider the mobile app as the most effective way to communicate with citizens and pointed out the importance to integrate their local systems with European services such as EFAS 3 and EFFIS 4 . The aggregated stakeholders' needs are ranked in Figure 3.
Data Scouting. We observed a very high productivity in this phase, obtaining 120 unique data collected. We organized the collected information into two main clusters: • WHO, which contains all the information related to the data providers (reporters). While the personal identity of the reporter is not required, the user role (e.g. citizen, first responder, organization) and its health status are of great interest because decision makers should have some element to evaluate the trustworthiness of the received information. Also information about the reporter safety are grouped in this cluster (e.g. health status, people in danger) • WHAT, which groups all information related to the context, that we classify in the following categories: "hazardrelated" (e.g. water level, fire presence, direction of wind), "damaged infrastructures" (e.g. a collapsed building), "available resources" (e.g. nearest safe place, drinkable water, connectivity). As shown in Figure 4, almost half (45%) of the information has been marked as priority data, out of which one third could be also provided by citizens. This demonstrate that also "common" people can have a not negligible role in providing critical data. As expected, first responders are considered the most trustworthy source, especially in relation to specific hazard related measures that require training to be properly estimated. We plot in Figure 5 the amount of unique data assigned to each phase, in percentage and divided by categories. Overall, the Response is the phase that requires a greater number of crowdsourced data, followed by the Preparedness and finally by the Post-disaster phase. The most numerous category is the "hazard-related", which have the biggest share in the Response phase. The same phase sees a major role of "available resources", and "reporter". "Damaged Infrastructure" is also quite wide, especially in the post-disaster phase. Although important, "safety" related data did not have many data.
User-Generated Prototypes. We isolate and separately analyze all the UI patterns generated, as suggested in [28]. The visual analysis highlights both commonalities and differences in terms of outlined functionalities, inserted data, layout and structure of the screen, as well as labeling. A first distinction is between: • An informative level provided by a centralized system and displayed on the mobile device. Such information is generated by the authorities managing the disaster and includes warnings, maps (hazard, risk, delineation, nowcasts, forecasts), and tasks for professional users; • A generative level, enabled by the crowdsourcing features, which refers to the information gathered from the field and to be routed to the control room. The visual analysis of the sketches allowed us to produce an overview (see Figure 6) showing the distribution of informative features provided by the system (green areas in Figure 6) and crowdsourcing features (dotted line in Figure 6). It is possible to observe that in general the informative level is always placed in the upper part of the screen, while the crowdsourcing functionalities are shown in the lower one. Apart from one singular sketch, a large section of the screen, from 1/2 up to 3/5, is reserved to manage the generative level.
Discussion and Consolidation of Outcomes. The final discussion of participants confirmed their great interest in the adoption of new technologies to enable the real-time transmission of risk management and mission critical information from and towards the field. Despite connectivity issues that may arise during disasters, cellular mobile devices are considered an highly reliable communication tools because of their ubiquity and capabilities in terms of geo-localization and provision of multimedia contents. At the same time, new technologies are also perceived as an additional problem to manage, due to the great effort required to get them adopted as an operational instrument, especially from an organizational point of view. The introduction of new technologies requires a training effort involving the staff and the public, in particular to implement collaborative paradigms like crowdsourcing. On one hand, mobile applications allow the collection of a large amount of valuable information, by both citizens and first responders. On the other hand, the receivers are forced to analyze such information in real-time and to implement some validation mechanisms, which can require resources in terms of time and staff. This points to the need of having automatic validation techniques, based on intelligent algorithms, in order to reduce the amount of data to be manually validated.

CONCLUSIONS AND FUTURE WORKS
The implementation of the proposed methodology based on the co-design framework resulted to be very effective in order to define key guidelines for the design of a crowdsourcing solution for DRR scenarios. We note that, in order to be effective in a complex field like DRR, it is important to plan the co-design within a preparatory phase, dedicated to share a common language and clearly set the basic concept definition. To be successful, each step of the proposed methodology has to be carefully implemented, defining a fixed time-schedule, the group composition, and the tools to be used in each phase. The introduction phase was essential to break the ice and put in evidence the richness as well as the experiences available within the involved participants, providing a sense of confidence that can inspire subsequent phases. Both the data scouting and the collaborative sketching were effective in outlining the crowdsourcing solution step by step, starting from the definition of the initial "bricks", i.e. the data to be collected and managed, up to the realization of a prototype aimed at a specific DRR scenarios. The final discussion enabled the consolidation of key concepts, and there were many fruitful exchanges of interesting ideas among designers, developers, experts, end-users and professionals of DRR. Furthermore, the participatory co-design framework gave sufficient space to all participants, allowing them to bring their own perspectives, to gain insight views and provide quick feedbacks that can be integrated in the subsequent design process. As a result, both pros and cons of the crowdsourcing solution could clearly emerge.
Following the outcomes for the co-design process, we will design and implementation the I-REACT mobile application, which is aimed at implementing the crowdsourcing of in-field reporting in DRR scenarios. We will evaluate the usability of the mobile application with real end-users (civil protections, volunteer organizations), assessing the compliance of the implementation with respect to their expectations.