Evaluation of mobile-based public participation in China's urban planning: Case study of the PinStreet platform

With the emergence of information technology, mobile-based public participation (MPP) has been introduced into urban planning practices. How to evaluate this new approach of public participation is important for supporting decision-making process and achieving effective public participation. Integrating traditional evaluation criteria and Internet product operation strategies, this study proposes an evaluation framework for MPP, which includes seven indicators from three dimensions: preparation, process and outcome. Taking PinStreet, a widely-used MPP platform in China, as a case, this study evaluates the effectiveness of seven pilot projects with a variety of themes and scales. The evaluation results show that factors lying in the aspects of environment, participant and organizer have a significant impact on the effectiveness of MPP. Finally, policy suggestions are proposed to promote the development of MPP.


Introduction
Public participation is important for making good urban planning decisions and has been widely adopted in Western countries since the late twentieth century. Such participation can be traced to the 1947 Town and Country Planning Act in England, which allowed citizens to express their opinions on large-scale developments after the Second World War. Over the next several decades, public participation was extensively explored theoretically (Arnstein, 1969;Davidoff, 1965;Forester, 1999;Friedman, 1973;Healey, 1992;Healey, 2005;Innes, 1995). Various approaches to public participation have also been developed in urban planning practices, such as referenda, public hearings, and consensus conferences (Rowe & Frewer, 2000). Public participation, rather than technical documents, has become an essential part of urban planning procedures and urban policy making.
In China, however, public participation has tended to be relatively passive, inadequate, and dispersed (Chen, Xiao, Chen, & Jiang, 2020;Enserink & Koppenjan, 2007;Hensengerth & Lu, 2019;Zhou, Hou, Yang, Chong, & Moon, 2019). For example, notices about major planning projects are more like "notifications" than "negotiations." Public hearings, questionnaire surveys, and interviews, mainly led by governments or elite planners, inevitably have problems such as insufficient representation and inadequate discussion. Other approaches, such as the "mayor's hotline," do not lead themselves to government decision-making since opinions and complaints are too dispersed to promote planning implementation. Given China's fast urban development and huge population, it is challenging for the Chinese government to implement public participation fairly and efficiently.
Because of their advantages in terms of interactivity and fast dissemination, Internet technology and new media have created potential opportunities for public participation in China. As of 2019, China has more than 800 million Internet users-perhaps the largest share in the world-and it has incubated many advanced Internet technologies, such as Alipay. Many programs for mobile-based public participation (MPP) have also come into being, becoming popular tools in urban planning policy making. PinStreet, for example, has been applied in 17 cities and has collected more than 120,000 opinions about urban and rural planning from citizens. However, the effectiveness of this emerging approach to public participation has hardly been investigated. A better understanding of the effectiveness of various application scenarios for MPP could help to optimize implementation procedures to achieve true public participation.
Against this background, the present study uses seven pilot projects based on a representative MPP platform in China-as cases to examine the effectiveness of MPP approaches. The intention of this study is to evaluate MPP and to reveal the difference of MPP performance across diversified project contexts as well as the influencing factors behind, rather than to compare the effectiveness among different forms of public participation. This study thus aims to answer the following research questions: (1) How can we systematically and quantitatively evaluate MPP, a new form of public participation in the new-media era?
(2) How does MPP perform in China across different project types and urban contexts, and what are the key factors influencing the effectiveness of MPP? The rest of this paper is organized as follows. Next, the literature review summarizes different forms of public participation as well as indicators for assessing the quality of civic involvement. The third section proposes an evaluation framework and relevant indicators for MPP. In the fourth section, the Pinstreet platform and seven pilot projects are introduced. Next, the evaluation results and influencing factors are analyzed. The last section concludes the paper and provides policy suggestions.

Forms of public participation
Public participation can be generally divided into three categories according to the evolution process. The first is traditional public participation (TPP) in which entry relies on having a substantial venue (e.g., public hearings, consensus conferences, advisory committees). The second and more modern category is Web-based public participation (WPP), which moves beyond spatiotemporal limitations by using technological tools (e.g., computers, phones, and tablets) to collect public opinion (Ertiö, 2018;Stern, Gudes, & Svoray, 2009). As for WPP, initially, personal computers and Internet access were explored. Currently, however, there is a focus on mobile devices, which can increase the scale of participation by augmenting real-time and place-based relations via wireless technology (Höffken & Streich, 2013). This is commonly defined as the third phase: mobile-based public participation, namely, MPP (Ertiö, 2013;Höffken & Streich, 2013).
In the pre-Internet age, people engaged in public affairs offline. The "most formalized public participation methods" of the past included referenda, public hearings, negotiated rule making, consensus conferences, and citizens' juries/panels (Rowe & Frewer, 2000). In most Western countries, various deliberative and collaborative mechanisms for traditional participation have been widely used, significantly affecting policy making. In China, however, the application of TPP has been less frequent and greatly simplified. Public hearings or consultations, which are used in an ex-post rather than ex-ante fashion, are the most popular ones.
Since traditional participation methods received some criticism, WPP has emerged to develop alternative and more interactive methods. One of the early attempts to make participation digital was an expertled geographic information system (GIS) for citizens (Ertiö, 2018). Later, Web-based GIS, as a spatial analysis tool, became connected to the Internet and introduced into the process of public participation. Most Western countries have been using WPP since the mid-1990s to gather residents' perceptions of the physical environment for the purpose of urban planning and have sought to design more user-friendly Web interfaces. Maptionnaire, for instance, has been used in projects in the US, Japan, and the Nordic countries. Another tool, the GeoCitizen 1 platform, facilitates the generation and discussion of ideas to enhance community surroundings, allowing for citizen participation. Some practical results show that WPP is useful for communication, and it may support participatory urban planning (Bugs, Granell, Fonts, Huerta, & Painho, 2010). However, when the concept and methods of WPP were introduced in the early 2000s in China, there were only a few instances of adoption, since it was not easy-to-use for people with inadequate technology skills.
In the era of Web 2.0, the media of citizen involvement have shifted from computer-based Internet to mobile devices and mobile-based social networking platforms. The popularity of Web 2.0 tools (e.g., blogs, Facebook, and WeChat) via mobile devices (e.g., smartphones and tablet computers) enables one-to-many and many-to-many communication channels, extending the scope of citizen participation. Today, mobile phones are ubiquitous. In China, for instance, there were almost 854 million mobile netizens in 2019, according to the China Internet Network Development Report. 2 As such, many city administrative departments and planning institutions have built mobile applications around public issues as well as urban problems; these are referred to as "civic apps" or "reporting apps" (Townsend, 2000). Examples of such apps include SeeClickFix 3 in the US, FixMyStreet 4 in the UK, and PinStreet 5 in China. These MPP instruments can help residents improve their neighborhood environments by "situating engagement" (Korn, 2013) when people are close to current infrastructure issues. Citizens typically use these apps to report nonemergency problems (e.g., street lighting, potholes, and fly tipping) observed while people are walking or commuting to work. The interfaces of these apps make participation simple and quick by automatically mapping location coordinates (via the phone's GPS sensor) and enabling photo attachments (camera sensor). Once reported, information is aggregated and analyzed to provide the city with real-time information to fix short-term problems and plan for long-term investments in urban planning.
Compared to traditional public participation, WPP and MPP perform better in terms of respecting local knowledge (Talen, 1999), supporting wider societal and environmental goals (Sieber, 2004), and addressing the diversity of data input (Ghose, 2001). Modern participation relying on the Internet and information technology is expected to profoundly affect development in the field of urban planning. In addition, compared to WPP, MPP has the characteristic of "pinning" citizens' knowledge to digital maps. It also makes better use of the advantages of new media and mobile communication while placing more emphasis on sharing and spreading knowledge in a "hybrid" place. Mobile participation allows for more factual and objective data collection through smartphones' sensors built-in (e.g., GPS, camera, and microphone) while previous participation could only gather the subjective experiences of a place.

Evaluation of public participation
Given the variety of patterns of participation mechanisms, the effectiveness of public participation is difficult to be evaluated (Rowe & Frewer, 2004). Arnstein (1969) noted that genuine participation can only be achieved via a high level of citizen empowerment. To further articulate the effectiveness of citizen involvement, some researchers-who generally focus on the characteristics or benefits of true public participation-have constructed evaluation frameworks based on two main types of criteria. The first is the process criterion; here, the evaluation explores how involvement exercises take place. A number of common themes arise in the process criterion: (1) sample validity: public participants should comprise a broadly representative sample; (2) early involvement: the public should be involved as early as possible; (3) participation cost: the public can freely participate at low or no cost; (4) independence: the participation process should be conducted in an independent, unbiased way; (5) transparency: input, feedback, and evidence of the analysis of the results should be presented in a timely manner; and (6) interaction: there are various methods for promoting exchange and communication; (Crosby, Kelly, & Schaefer, 1986;Godschalk & Stiftel, 1981;Halvorsen, 2001;Renn, Webler, & Wiedemann, 2013). The second type is the outcome criterion, which entails assessing the results of participatory processes. There are three subcriteria: (1) proposals generated: a considerable portion of high-quality ideas are generated; (2) consensus achieved: value consensus and interest consensus between different participants are achieved; and (3) influence imposed: there is evidence of increased publicity resulting in a wider adoption of proposals (Bickerstaff & Walker, 2001;Petts, 1995;Rowe, Horlick-Jones, Walls, Poortinga, & Pidgeon, 2008).
Although previous studies have proposed various indicators to evaluate public participation, some indicators, especially those of the process criterion, are no longer applicable to evaluate WPP and MPP. For instance, participation cost including time and finance is almost non-existent since a mobile phone with Internet connection is enough for the general public to post messages or images to express opinions openly and freely (Bugs et al., 2010). Additionally, civic apps usually take advantage of Internet resources and technology-such as social media, location-based service (LBS), and software as a service (SaaS)-to continuously optimize usability and expand user coverage, moving past spatiotemporal boundaries (Zhao, Lin, & Derudder, 2018). In this way, real-time information exchange and relatively more equal network structures can be achieved (Kingston, Carver, Evans, & Turton, 2000;Mandarano, Meenar, & Steins, 2010). Thus, evaluation indicators such as early involvement and independence are also no longer suited to WPP and MPP practice (Carver, 2001;Han & Peng, 2003).
In response to the new features of WPP and MPP, more targeted indicators have been orchestrated in recent studies. In terms of the participatory process, scholars pay more attention to the actual scope and frequency of online involvement, since people are theoretically allowed to take online participation anytime and anywhere. For instance, some scholars propose the "scalability" of online participation, the term reflecting the capacity for augmenting the number of participants and expanding a range of geographical areas (Nyerges & Aguirre, 2011). Based on the scalability criterion, Jankowski, Czepkiewicz, Młodkowski, Zwoliński, and Wójcicki (2019) address that effective online participation should have considerable capability to "scale up" and "scale out". On the other hand, Stern et al. (2009) argue that "repeated entry" to the web interface of involvement platform is also regarded as one of the indicators of effective WPP. From the perspective of outcome criterion, effectiveness of WPP and MPP, similar to that of TPP, can be indicated by consensus achieved and influence imposed (Sarno, 2005). As interactive social network is regarded as a salient feature of effective online participation (Pang, 2018), participants are connected in the network to comment and communicate on certain issues with the purpose of consensus achievement. Furthermore, the connection between participants and governments is built with the empowerment of the public to policy development in the decisionmaking dialog (Elwood, 2002;Sieber, 2006).
Although previous studies have articulated process and outcome of participation as two dimensions to evaluate the effectiveness of WPP and MPP, another important dimension, namely preparation of public participation, is neglected. Difficulty of access to online participation platform is often criticized as the deficiency of WPP and MPP, since it may not be easy-to-use for people with inadequate technology skills (Carver, Evans, Kingston, & Turton, 2001). Simply deploying an online platform without any guidance or mobilization can hardly constitute an effective online participation (Carver, 2003). In this case, the present study argues that the preparation of public participation including various ways of informing and guiding citizens to access online participation should be regarded as a salient evaluation dimension of WPP and MPP.

Research framework
Based on the general evaluation criteria of public participation and the specific characteristics of MPP, this study seeks to propose an evaluation framework for MPP with feasible measurement indicators, so as to systematically and quantitatively evaluate the effectiveness of this new public participation approach.

Evaluation framework of MPP
Using the Internet as the platform and smartphones as the medium, MPP methods share many characteristics with Internet products (e.g., user growth and closed-loop operation mechanism), therefore, it is preferable to integrate traditional evaluation criteria with evaluation strategies from Internet product theories. The famous "pirate metrics" (McClure, 2007), also known as the AARRR -acquisition, activation, retention, referral, and revenue -has been widely accepted, with its five key indicators for measuring the effectiveness of user growth tactics. Specifically, acquisition refers to different channels of guiding potential customers to a particular app; activation relates to that users take the first visit to the app interface; retention means that users visit the app for multiple times; revenue implies that users purchase service or products in the app; and referral indicates that users like the app enough to refer it to others.
In large part, AARRR matches the operation logic of MPP for reaching potential stakeholders in the preparation stage (acquisition), motivating them to participate and keeping them active in generating ideas during the participation process (activation and retention) and purchasing service as the outcome (revenue). Although revenue is not a direct concern of MPP, governments and the public can both benefit from the expression of opinion and the changes implemented. As such, revenue in the context of public participation can be understood as a Pareto improvement that makes as many participating stakeholders better off as possible without imposing expert-controlled plans on the public against its will.
Therefore, an evaluation framework of MPP can be deduced from integrating three criteria of traditional evaluation and key strategies of AARRR model (Fig. 1). As for the preparation criterion, acquisition can be understood as the procedure for an MPP platform to reach potential users through on-site or online promotion and then turn them into actual users, which can be measured by user reachability rate and acquisition rate. Regarding the process criterion, activation and retention reflect growing public interest in the participatory process through submitting ideas to the MPP platform once or multiple times or contributing superior proposals, which can be measured by participation rate, reentry rate and superior contribution rate. As for the outcome criterion, since revenue can be understood as benefit to stakeholders, it indicates that effective participation will eventually lead to consensus and adoption, which can be evaluated by consensus rate and adoption rate, reflecting the meaning and significance of public participation.

Indicator measurement of MPP
According to the operation mechanism and data of civic apps, the evaluation indicators of MPP are measured via following methods (Table 1). In the preparation dimension, reachability rate is measured by the number of potential users reached by all promotion approaches at the target site. The volume of customer flow is estimated for on-site promotion according to the number and service level of the promotion spot and promotion hours. Meanwhile, for online promotion, the reading count of new media articles and the exposure rate are also taken into consideration. Based on the number of people reached, acquisition rate is calculated by the real increment of users in the MPP platform during the promotion period. In the process dimension, participation rate and reentry rate are measured by the proportion of users who successfully submitted proposals and the proportion of users who repeated submitted proposals on different days respectively. The limit of different days on reentry rate calculation excludes the potential bias wherein people tend to submit several proposals at the promotion spot to get more monetary rewards. In addition, a condition of "more than 30 Chinese characters" was set for the calculation of superior contribution rate to filter proposals with invalid content or low quality. In the outcome dimension, the "thumbs-up" (dianzan) function of many civic apps provides an effective channel for participants to support other people's proposals and express consensus. Therefore, consensus rate is calculated by the number of "thumbs-up" proposals over the total number of proposals. Finally, adoption rate is measured by the proportion of adopted proposals in all proposals. For public participation activities launched by local government with direct decision-making power, a considerable proportion of the superior proposals may be adopted and implemented in a short time; for those initiated by other organizers, the adoption rate may be low or temporarily not available.

Case study
This study choses PinStreet, a widely-used MPP platform in China, to carry out the evaluation analysis of MPP. Seven pilot projects with different application scenarios on the PinStreet platform were selected to objectively evaluate the effectiveness of mobile-based public participation. Although the PinStreet cases were exclusively used to elaborate the evaluation using the framework, this approach would be applicable to most MPP cases around the world.

Introduction of PinStreet platform
The PinStreet platform is the first and largest third-party online application dedicated to supporting public participation activities to improve urban quality in China. Since being launched in 2017, the PinStreet platform has been applied to 35 projects across 17 cities, collecting more than 120,000 individual proposals in China. It is built on a WeChat miniprogram platform with LBS technology. Integrated with WeChat, the most popular messaging app in China, the PinStreet platform is convenient and efficient to collect information and data from a large amount of potential user groups. With LBS, public advices can be correspondingly connected to urban spaces, thus providing more operational suggestions for the government (Figs. 2 and 3). The implementation of a PinStreet project is either location-oriented or topicoriented. Usually, each project focuses on several potential local issues for residents to discuss, such as shared bicycle parking, public transit, pocket park, and public service. Participants can express their opinions on the platform in a transparent way while avoiding public exposure or social pressure. Diverse means, such as mapping interactions, commenting and transmitting, and social media communication, are also integrated into the platform to facilitate interaction among users. After the collection of opinions, a data report is generated based on the live map and data analysis and is delivered to the government or planning institutes to help them identify main types and locations of urban problems and provide accurate and elaborate improvement schemes.

Selection of pilot projects
We select seven pilot projects using the Pinstreet platform with the consideration of the diversity of scales, location, and organizer types. As shown in Table 2, among the seven projects, five are at subdistrict scale with the population ranging from approximately 50,000 to 250,000, and two are at district scale with the population ranging from 400,000 to 900,000; two are in old town areas and five are in suburban areas; three are organized by government and four are organized by urban planning institutes. These MPP activities were carried out during the period of time from September 2017 to October 2018. The number of proposals collected in a single project ranges from 299 to 2117.  The real increment of users during the promotion period over the number of reached citizens Process Participation rate The proportion of users who actually submitted proposals Reentry rate The proportion of users who repeated submitted proposals on different days Superior contribution rate The proportion of proposals with more than 30 Chinese characters Outcome Consensus rate The proportion of proposals with a "thumbs-up" Adoption rate The proportion of proposals adopted and implemented by the government X. Wang, et al. Cities 109 (2021) Table 3 shows evaluation results for the seven above-mentioned PinStreet experiments based on the MPP evaluation framework and indicators (see the appendixes A and B for detailed calculations). The result of preparation criterion indicates that MPP in China has achieved good overall reachability, averaging 31%, but poor user acquisition, averaging 3%. The low conversion rate from potential users to actual users reflects the current participation situation in China where citizens show little interest in public participation in the early stage. Another criterion, process, obtains high values in participation rate (62% on average) and medium values in superior contribution rate (20% on average) but low values in reentry rate (9% on average). The three values imply some interesting phenomena in the participatory process. First of all, once people enter the MPP platform, more than half of them are willing to participate. 20% of the participants who have rich local knowledge are able to contribute superior proposals, which indicates the one-time participation is relatively effective. However, when it comes to continuous participation, few people develop the habit of using the MPP application, although they may use mobile phone every day. As for outcome criterion, on the one hand, the average value of the consensus rate is 12% which indicates the public comments are difficult to reach a consensus. On the other hand, it should be noted that decision-making process is mainly dominated by clients. Within the seven cases, only government-led programs achieve the transformation from proposals to real actions, although the adoption rate is still very low. In the Pingshan program, the adoption rate reaches the highest value (6%).

Preparation related criteria
Reachability rate is directly correlated to the scope of projects. This study finds that subdistrict-scale projects generally have higher reachability values, followed by district-scale projects. This is possibly caused by the smaller population base of subdistricts, which can be easily propagated to (Fig. 4).
The difference of acquisition rate is not obvious, all being in the range of 2% to 4%. This shows that although the number of reachable  X. Wang, et al. Cities 109 (2021) 102993 people is large, the amount of people who are truly interested in the problem or willing to pay ongoing attention to it is small. In addition, the dominant influential factor of this value can be abstracted to the identity of clients. Compared to projects entrusted to other clients, the ones commissioned by governments all achieve a 4% rate -easily promoted and able to attract more attention.

Process related criteria
In terms of participation rate, aside from the Huilongguan project, those with suburban-focused issues could achieve higher participation than those in old towns. This is partly because public infrastructures in suburban areas are not in high quality compared to those in old towns.
Most citizens living or working in suburban areas are troubled by such problems and are therefore motivated to give their opinions (Fig. 5).
Regarding to reentry rate, for old town projects, the reentry rate was 22% on average, which means more citizens were willing to give opinions more often. By contrast, suburban projects had an average of 4%. This is partly attributable to the population composition of local areas. In Beijing, elderly and natives account for a considerable proportion of the regional population in the old town. They make more use of public infrastructure and are more willing to pay ongoing attention to their local living environment, compared to young or nonlocal people who are highly mobile. Taking the Jingshan project as an example, Jingshan had a local population of 27,331 in 2017, occupying 62.5% of the total  Note: (1) reachability rate = total number of citizens reached/total population; (2) acquisition rate = user increment/total number of citizens reached; (3) participation rate = total number of users who have a proposal/user increment; (4) superior contribution rate = total number of proposals with more than 30 words/ proposal number; (5) reentry rate = total number of users who proposed more than once/total number of users who have a proposal; (6) consensus rate = total number of proposals with "thumbs-up"/proposal number; (7) adoption rate = total number of proposals adopted by government/proposal number. Fig. 4. Reachability rate in relation to scale.
X. Wang, et al. Cities 109 (2021) 102993 regional population -far above the average proportion of local residents in Beijing (45.6%). 6 Residents living in this area hope for continuous improvements to their community environment; thus, most of them give opinions more than once, achieving a higher reentry rate (26%) (Fig. 6).
With regard to superior contribution rate, the average value is 22% with the range from 17% to 24%. No obvious patterns can be observed across categories of projects. As for Xinjiekou which got the highest rate (24%), it might be because elderly and natives in the old town are more familiar with local problems than those in other areas, thus can provide more details in their proposals. The other high rate project is Qinghuayuan (23%). Its reason might be that Qinghuayuan subdistrict covers the campus of Tsinghua University, consisting of a large proportion of well-educated population, and thus they are more likely to contribute superior proposals.

Outcome related criteria
Consensus rate shows that the factor of project location has a significant impact on the proportion of consensus achieved. Due to the larger proportion of local residents in old towns (e.g., Jingshan, Xinjiekou), an "acquaintance society" is more likely to form, where consensus can be easily achieved by opinion leaders' advocacy, daily communication, and social initiative. An exception is Qinghuayuan. With a majority of population of students and faculty from Tsinghua University, Qinghuayuan is as well connected a community as those in old towns, and thus consensus rate is also high (Fig. 7).
In terms of adoption rate, there is an outstanding influential factor: the character of the client. The data shows that projects with the government as the client are more likely to get responses and be practically implemented. Since, for example, the participation scope of the  X. Wang, et al. Cities 109 (2021) 102993 Pingshan project corresponds to the jurisdiction of the Pingshan district government, it had the highest adoption rate. By contrast, without practical support from local government, public proposals are hard to put forward and translate into implementation.

Influencing factors analysis
Effectiveness evaluation is employed not only to sum up the effect of previous participation practice, but also to address ways guiding the next round of participation (Kaiser, Godschalk, & Chapin, 1995). Therefore, to identify the key factors affecting respondents' willingness to participate and further influencing the outcome of participation is very important. Combining previous literature and the above analysis based on seven pilot PinStreet projects, we further examine the difference across MPP cases following the line of environment, organizer, and participant fronts.
First, the effectiveness of a participation practice is largely influenced by environmental factor, particularly referring to the scale and geographic extent (Carver, 2003). As stated by Burton (2009), an effective citizen participation can only be local since the burden of becoming involved can be overcome only by those who are affected most by the issue at hand. Likewise, Carver et al. (2001) argue that at local scale, the greater majority of citizens are interested in public affairs that pertain to their area, while as spatial scale increases to regional or national scale, less people are engaged.
Consistent with previous studies, this study finds that environmental factors are crucial in the context of online public participation. In terms of scale, it is more effective to organize a participation practice at subdistrict scale than that at district scale, given the promotion cost to reach potential participants in the early involvement stage (Fig. 4). What previous studies have not clarified is that the quality of living environment, such as municipal facilities, recreation facilities, and street design, has great influence. Specifically, the willingness to participate for the first time is mainly determined by participants' living conditions since it is more like an immediate public response triggered by an unpleasant living environment. For example, when the participation site is situated in a suburban area like Qinghuayuan, Qinghe, Pingshan and Tongzhou, where public infrastructure and residential amenities are weak, the public is more willing to participate. Huilongguan with a low-level participation rate is an exception. A possible reason is that its on-site promotion was not held in shopping malls, but near subway stations where most commuters passing by were in a hurry and only a small proportion had spare time for comments. (Fig. 5).
Second, organizer factors, including a set of institutional elements such as government mobilization, information publicity systems and decision-making mechanisms, affect public participatory initiative. Participation exercises rely heavily on government organization, especially in China. The fact is that if exercise is not initiated by local government but rather other organization which has no voice in decision making, people's participation confidence would be decreased (Brovelli, Minghini, & Zamboni, 2016). Nevertheless, a vast literature argues that official factors impose negative effects. Evidence from Jupp's fieldwork shows that government-initiated public participation is often seen as ineffective (Jupp, 2007). Swapan (2014) demonstrates that diminishing level of trust in official agencies prevents the general public from participating. Xu (2018) argues that government mobilization will lead to weak public autonomy and participation enthusiasm, which cannot be called real "participation" and is not conducive to the growth of civil society.
Our results, however, suggest that public participation mobilized by the government may not necessarily inhibit the growth of civil society. In fact, the role of governments in the Jingshan, Qinghe, and Pingshan projects was more like that of an "activator of public participation" or a "provider of participating channels" than that of a "controller." In the context of immature social self-mobilization in China, public participation by virtue of government mobilization represents a feasible mode of social governance combining top-down and bottom-up aspects (Chen et al., 2020). Moreover, the roles of organizers should not be ignored since they significantly influence the eventual effects of public participation. If the organizer (government) has direct administrative and decision power over the project site, public proposals are more likely to get responses and be adopted for implementation. The Pingshan project provides strong evidence for this.
Third, a commonly accepted factor is about participants' attributes, such as age, education level, economic condition, and sense of community. According to Xie's research, which investigates participatory practice in rural China, the public with low levels of literacy are incapable of participating effectively and actively (Xie, 2016). Meanwhile, young people who have easy access to online participation are more likely to contribute effective proposals in the participatory process (Stern et al., 2009). In addition, studies also indicate that people who live in the community longer tend to draw on more local knowledge and feel more sense of community, and thus make more efforts to participation (Han, Shih, Rosson, & Carroll, 2014;Rafaeli, Ariel, & Hayat, 2005). X. Wang, et al. Cities 109 (2021) 102993 Our results reveal another two participants' attributes, direct interest links and lay knowledge, are helpful to promote sustained participation and broad consensus. On the one hand, participants who are long-term residents have a strong desire for a livable community; therefore, they can keep participating until the environment improves (Fig. 6). On the other hand, people with local knowledge are more likely to achieve consensus, largely because they are quite familiar with their living environments and have the similar sense of living experience (Fig. 7). Most participants from the two old towns (Jingshan and Xinjiekou) have both interest-related and knowledge-related attributes, thus showing the higher reentry rate and consensus rate.

Conclusion and discussion
With the spread of Internet technology and new media, MPP offers a potential approach to support the decision-making process of urban planning in addition to TPP and WPP. Thanks to the characteristics of low cost, independency and transparency, MPP is especially suitable for countries with a high proportion of Internet users and smart phone users like China. In order to evaluate the effectiveness of this new approach, this paper proposes an evaluation framework for MPP including three dimensions and seven indicators by integrating traditional evaluation criteria and Internet product operation strategies. Seven pilot projects based on a representative MPP platform in China are chosen to test the feasibility of the framework and indicators. Some conclusions can be drawn with regard to the research questions on the evaluation criteria and influencing factors of the effectiveness of MPP.
First, as for the preparation dimension of the evaluation framework, we find that MPP has relevantly high reachability rate (31%) but low acquisition rate (3%). This shows the two sides of the widespread dissemination feature of MPP. Although MPP has a broader ability to spread beyond physical boundaries than TPP and WPP, it's more difficult to transfer potential users to actual users in the open and hybrid space of MPP than in the substantial venue of TPP. Moreover, the difference of reachability rates between district-scale projects and subdistrict-scale ones shows that the larger the scale, the lower the reachability rate; therefore, it's important to find a balance between population size and reachability rate under the constraint of promotion cost.
Second, regarding the process dimension, MPP's participation rate is quite high (62%), while the superior contribution rate and reentry rate are relevantly low (20% and 9% respectively). These indicators are unique for MPP since it provides an open, free and convenient platform for participants to express opinions. The significant disparity of participation rates and reentry rates between old town areas and suburban areas shows the mixed impacts of environment' and participant' attributes on the effectiveness of MPP. Unpleasant living conditions in suburban areas facilitate residents' passive one-time participation, while long-term residence with strong community attachment in old town areas promote residents' positive continuous participation.
Third, as for the outcome dimension, we find that MPP's consensus rate is 12% in all projects and adoption rate is 4% in government-led programs. The difference of consensus rate between old town and suburban areas indicates that interaction between participants is influenced by the acquaintance level of the community. Regarding the adoption rate, our findings show that the adoption rate is highly related to the government involvement in Chinese context. MPP is helpful for integrating top-down management and bottom-up participation by enlarging the participating channel for citizens. Our findings also provide some policy implications for MPP in urban planning practices. First, in the preparation stage of MPP, both online and offline promotion approaches are strongly encouraged to reach as many potential participants as possible without ignoring those with inadequate technology skills or excluded by online dissemination channels. Second, in the process phase, longer activity time and setting offline information center would help to improve reentry rate. Third, facilitating a direct dialog with decision-makers is conducive to empowerment. When the participation scope is within the jurisdiction of government, the adoption rate would be raised and effective public participation would be more promising to be achieved.
This study has some limitations. First, the PinStreet cases collected limited information on participants and lack detailed demographic data, such as age, gender, occupation, and income. Therefore, we cannot evaluate the potential effect of participants' profiles on MPP effectiveness. Second, the interactions among participants, as well as between participants and the government, were not quantified and integrated into the framework due to a lack of data, which remains to be improved in future mechanism designs and software iterations. Third, as MPP in urban planning practice is still at a preliminary stage, with the increase of cases, more empirical studies are warranted to deepen the understanding of the effectiveness of MPP and to compare the difference among MPP, WPP and TPP.

Declaration of competing interest
None.