Searching for the Whole Truth: Harnessing the Power of Intellectual Humility to Boost Better Search on Debated Topics

We often use search engines when seeking information for opinion-forming and decision-making on debated topics. However, searching for resources on debated topics to gain well-rounded knowledge is cognitively demanding, leaving us vulnerable to cognitive biases, such as confirmation bias. This can impede well-informed decision-making, and on a societal level, snowball to compel extremism and polarization. Most existing approaches to support better search apply nudges that directly modify user behavior. Such interventions bear the risk of harming user autonomy. Here, we discuss the shift we envision towards autonomy-preserving interventions that boost users’ metacognitive skills, specifically their intellectual humility (IH)–the ability to recognize the fallibility of one’s beliefs and the limits of one’s knowledge. While simple interventions to boost IH have shown promise, the effect on users’ search behavior has yet to be investigated. We present critical research questions, challenges, and an initial research plan to advance knowledge in this area.


INTRODUCTION AND MOTIVATION
Search engines are widely used to locate information-from a simple definition of a word to resources that are then used to inform searchers' opinions or gather advice before making a decision on debated topics, issues of ongoing discussion, such as should people become vegan?(see Figure 1).Regardless of the aim of the search, according to Kuhlthau's model, the information search process (ISP) involves six stages: Task initiation (e.g., reason to search for information), topic selection (e.g., setting a search goal), pre-focus exploration and focus formation (e.g., querying and browsing the results), information collection (e.g., engaging with search results), and search closure (e.g., analysing and synthesizing information) [27].When focused specifically on debated topics, we posit that the ISP is particularly cognitively demanding since it involves issuing unbiased queries, browsing vast amounts of retrieved resources, learning about an often complex subject matter, accepting a certain level of uncertainty, and thinking critically to objectively assess information, even if the topic is emotionally charged and might pose a threat to personal values.Consequently, a user might experience cognitive biases, such as confirmation bias -the tendency to favor information that confirms prior attitudes, beliefs, and values [35], during the search process [3,14,19,36,41].This can impede the user from making well-informed decisions [36,46], and in a societal context, be a source of increasing extremism and polarization [19,30].
Existing confirmation bias mitigation approaches nudge users towards engagement with attitude-opposing viewpoints, such as preference-inconsistent recommendations [44], alternative query suggestions [40], and warning labels and obfuscations of attitudeconfirming search results [43].These interventions are crucial first steps toward addressing the risks of confirmation bias during web search.However, the interventions themselves cause undesired sideeffects: Nudging strategies that tap into the automatic mind by modifying the ease of accessing some information (friction) harm user autonomy [7,17,32], cause reactance such as decreased exploration [7,40], prevent the detection of incorrect applications [43], and can trigger a feeling of being censored [48].Thus, such nudging approaches are criticized for the risk of paternalism, enabling manipulation with malicious intentions (e.g., censoring), and the lack of learning [17,21,32].Further, these interventions focus on specific search behaviors (e.g.clicks on attitude-opposing search results).This scope is often too narrow and does not capture the full complexity of the broader problem: Confirmation bias can impact the users' search behavior during the whole search process, from querying to assessing and remembering arguments made in the retrieved documents [3,50].Consequently, search behavior should be evaluated in a more comprehensive manner, for example by identifying whether users engage in exploratory or lookup search [2], or what search roles (e.g., non-motivated searchers who stop at the first result, confident and competent power searchers) they take on [22,28] after being exposed to different interventions.
We posit that the value of the aforementioned interventions in real-world settings is somewhat limited when considering the risk of associated side-effects.This calls for the need to explore alternative methods that can sustainably guide searchers to responsibly engage with resources on debated topics.We propose to shift confirmation bias mitigation efforts towards enhancing users' meta-cognitive abilities that lead to less biased behavior.Particularly vital for confirmation bias mitigation is users' intellectual humility (IH) [10,38,39].IH describes a metacognitive core consisting of recognizing the limits of one's knowledge and being aware of the fallibility of one's opinions and beliefs [37].Researchers have successfully achieved temporary boosts of IH, for instance, simply by informing participants about the benefits of high IH [37,39].This is yet to be explored in the context of web search.Boosting IH to indirectly modify user behaviour instead of nudging user behaviour directly would avoid harming users' autonomy, be less prone to abuse and errors, and tackle the risks of confirmation bias and other factors that impede good search behavior more comprehensively and sustainably [18,24,32].
With this work, we present and motivate our vision of autonomypreserving interventions to support better search behavior.As captured in Figure 2, interventions boosting users' IH (meta-cognitive state) could empower users to explore and engage with different viewpoints (search behavior), critically assess the encountered information when forming opinions and making decisions (search consequences), and ultimately contribute to a more inclusive and tolerant society where search engines equip and encourage individuals to gain well-rounded knowledge on debated topics before forming opinions or making decisions.

RELATED WORK
In this section, we discuss background literature concerning different areas of research that form the basis for the proposed vision.

Debated Topics and the ISP
Debated (controversial/disputed) topics are subjects of an ongoing discussion on which individuals or groups do not have the same opinions.While some debated topics are extremely one-sided and supported by solid scientific evidence, others are less settled because there are reasonable arguments on either side.People use search engines to find information on debated topics, for example, to form opinions or gather advice before making a decision [8,49].This would imply that the objectives for this search task are (1) gaining a well-rounded understanding of the topic and the different arguments by gathering information and (2) forming opinions or making decisions in response to all the collected information.
Achieving these objectives requires exploratory search that is focused on investigation and learning, as opposed to lookup search aimed at retrieving facts to answer specific questions [33].Exploratory search activities are complex and perceived as challenging, since they require users with different needs and abilities to critically analyse, synthesize, and evaluate information [51].Moreover, searchers can experience a certain level of threat when seeking resources on debated topics: Strong prior beliefs and convictions, as well as political, religious, or ethical values, are likely to be challenged by attitude-opposing viewpoints [39].
According to Kuhlthau's ISP model [27], searching for information involves six stages.The particularly high level of cognitive demand imposed on searchers becomes apparent when contemplating what these stages encompass for web search on debated topics.
1 Task initiation is often prompted when users need to form their attitude on a topic or seek advice before making a decision.Such situations can increase the stress perceived during the ISP, e.g., through time pressure, uncertainty, and the complexity of the subject matter of many debated topics [36]. 2 Topic selection compels users to approach the search task with the goal of gaining knowledge and being well-informed about the subject matter instead of finding evidence to support their prior attitude.3 & 4 Pre-focus exploration and focus formation require users to formulate unbiased queries, and navigate and accurately assess the quality of vast amounts of resources.5 Information collection takes place as the users engage with and evaluate different viewpoints and arguments to extend knowledge on and gain a well-rounded understanding of the often complex subject matter.6 Search closure results from users thinking critically to objectively assess the information they have encountered to be able to make well-informed decisions, even if the topic is emotionally charged and alternative viewpoints threaten users' prior beliefs, convictions, and values.
To deal with the complexity, cognitive demand, and uncertainty of web search for resources on debated topics, users are likely to apply strategies to simplify the search task [3].A dominant strategy to limit the uncertainty and amount of possible resources to engage with is the human tendency to prioritize information that confirms prior attitudes when searching for, engaging with, and assessing information (confirmation bias) [35].
Confirmation bias during search on debated topics can occur at different stages of the ISP, e.g., when employing positive test strategies when querying (focus formation, see Figure 1), clicking primarily on attitude-confirming search results (information collection), and actively disregarding information that opposes users' prior attitudes when assessing arguments to form an opinion or make a decision (search closure) [3,47,50].Thus, confirmation bias in this case would either limit exploratory search behavior to merely one-sided information or prevent all exploration by causing users to engage in lookup search behavior that aims at retrieving facts to support their prior attitude.In addition to confirmation bias, users' exploration behavior of resources on debated topics can be further inhibited by manifold other cognitive biases that can occur during search [3], or by external obstacles, for example, search engine result pages (SERPs) with viewpoint-biased rankings [12,13], or interfaces designed to steer behaviour for commercial gain [24].Lacking exploration of different viewpoints has negative consequences on the quality of attitude-forming and decision-making since it prevents users from building well-rounded knowledge [3,36].This has been linked to increased polarization and ideological extremism, thus additionally harming the quality of public discourse [19,30].
In summary, seeking resources about debated topics in the pursuit of well-informedness is a non-trivial undertaking; it requires complex search behavior known to be cognitively taxing throughout the ISP.This high demand makes users vulnerable to cognitive biases, has a detrimental impact on all ISP stages, and hence prevents users from becoming well-informed, which has been linked to increased polarization and extremism and thus should be mitigated.

Intellectual Humility
According to the definition that Leary et al. [29] developed following discussions with an interdisciplinary group, the metacognitive core of intellectual humility describes people's "recognition that a particular personal belief may be fallible, accompanied by an appropriate attentiveness to limitations in the evidentiary basis of that belief and to one's own limitations in containing and evaluating relevant information".Porter et al. [37] have synthesized the common thread within different definitions of IH from various fields and suggest that it encompasses (1) recognizing the fallibility of one's beliefs, and (2) recognizing the limits of one's knowledge.
In the context of knowledge and information behavior, IH can counter typical behavioral patterns that searchers tend to exhibit when subjected to confirmation bias.Intellectually humble people are more likely to indulge in increased information seeking and have a high motivation to gain new knowledge [15,25,38].They also spend more time learning about opposing arguments and reading information countering their prior beliefs [5,11,38].High IH enables people to distinguish the strength of different arguments, even those opposing their prior belief [29].It can also help users overcome external obstacles that impede optimal search behavior, e.g., by decreasing their susceptibility to false and misleading information [6,23].However, searchers are less likely to exhibit IH when engaging with a topic on which they have a strong prior opinion, and/or when their political, religious, or ethical values appear to be challenged [26].
IH has been identified as a relatively stable trait (a person's general level of IH).Yet, researchers have observed substantial and systematic within-person variability of IH as a state (a person's level of IH in a specific context) [37,52].IH on the trait-level positively correlates with other user traits, such as the need for cognition and cognitive reflection [25,29,38], or open-minded thinking and curiosity [26,29,53].Further, people exhibiting behavior related to high IH are likely influenced by their cultural background.Someone living in an environment that requires high social coordination is more likely to be intellectually humble than someone who lives in an individualistic environment [16].
Researchers have developed several methods to measure IH which differ in type (questionnaire vs. behavioral task), the aspects of IH they emphasise (limits of knowledge, fallibility awareness), whether they measure IH on the trait-or state-level, and the assumed dimensionality of IH (up to four) [37].Alfano et al. [1] developed one of the most extensively tested measures of IH.Their scale captures the trait-level of IH on the four dimensions of open-mindedness vs. arrogance, intellectual modesty vs. vanity, corrigibility vs. fragility, and engagement vs. boredom.However, questionnaire-based measures of IH on the trait-level have been criticised for being vulnerable to social desirability bias, and for failing to detect context-and intervention-dependent variability [37].Behavioral-task based measures cannot be distorted by self-report biases.Still, they might only capture a segment of artificial behavior, induced by the experimental setting.Porter et al. [37] suggest applying questionnaire-based measures and asking people to recall a specific situation when filling the questionnaire or to measure the trait-level by repeated measures of the state-level of IH to mitigate response bias.To sidestep issues related to questionnaire-and behavioral task based measures, Christen et al. [9] have investigated an indirect method of assessing IH with natural language processing (NLP) techniques to extract different dimensions of IH from written text.
Summarizing the findings on IH, this metacognitive concept encompasses recognizing the fallibility of one's beliefs and the limits of one's knowledge.High IH was found to counter behavioral patterns during information seeking that are common for confirmation bias.Substantial within-person variability of IH suggests that it can also be considered a context-dependent state.Researchers have developed many methods to measure IH on the trait-and state-level, such as multidimensional questionnaires or behavioral tasks, which have different advantages and disadvantages.

Mitigating Cognitive Bias during Search
To support better search behavior, enabling well-informed attitudeforming and decision-making, Lorenz-Spreen et al. [32] propose effective web governance through the application of behavioral interventions in form of nudging or boosting.This is in line with the guide to cognitive debiasing by Soll et al. [45], who suggest either to modify the environment or to modify the user.So far, approaches to mitigate confirmation bias during web search have focused on modifying the environment, e.g., with preference-inconsistent recommendations [44], alternative query suggestions [40], and warning labels and obfuscations of attitude-confirming search results [43] to nudge searchers towards increased engagement with attitudeopposing viewpoints.
Interventions that nudge user behavior by modifying the search environment and ease of access to different search results are nontransparent and target automatic thinking processes that can harm user autonomy [7,17].This indicates that the decision of what information users engage with is, without users' awareness, not entirely theirs.This non-transparency can result in users being unable to detect incorrect applications of the nudge [43], enables concealed applications of the approach with malicious intentions, or triggers a feeling of being censored [48].
Boosting interventions that modify the user, generally preserve user autonomy by aiming at fostering people's existing cognitive or motivational competencies, thus encompassing a learning component and, unlike nudging, remaining effective even after the intervention [18,32].A promising metacognitive concept for mitigating confirmation bias is IH, the variability of which as a contextdependent state generates opportunities for interventions that attempt to boost it.Researchers have explored different strategies that temporarily boost IH on the state-level [37].For instance, asking participants to reflect on scenarios from a self-distanced perspective [16], quizzing participants on a topic to make them realize the limits of their knowledge [20,34], or simply informing them about the benefits of IH [39].Krumrei-Mancuso and Newman [26] observed that approaches to boost IH, such as asking participants to complete a short IH scale, might require personalization to be effective for all users.The authors found that priming IH increased responsiveness to information on a debated topic in high IH users while for low IH users, the priming did not have an effect.
In summary, while effective, existing approaches against confirmation bias during search that apply nudges to directly modify user behaviour, run the risk of harming user autonomy.This motivates our quest for alternative interventions that aim at boosting users' metacognitive states.Boosting approaches that aim at fostering metacognitive concepts could be an autonomy-preserving alternative.For that, IH is a particularly relevant concept since high IH on the trait-level was shown to counter confirmation bias during different stages of the information seeking and attitude-forming process.The substantial within-person variability of IH as a contextdependent state generates opportunities to boost IH, which have been applied successfully in non-search contexts.

VISION: BOOSTING IH TO MITIGATE CONFIRMATION BIAS
In the context of bias mitigation, non-transparent and automatic nudging approaches can be seen as a form of paternalism, as they suggest that users' behavior is faulty and requires (manipulative) correction.Boosting approaches that target metacognitive skills, on the other hand, view users as individuals who carry the competencies and motivation needed to overcome their biases within them and, from that, can develop them [32].To avoid the risk of harming user autonomy, we propose developing interventions that boost the metacognitive state of searchers (see Figure 2).Such boosts could positively impact users' overall search behavior and the search consequences and mitigate their confirmation bias throughout the ISP, from setting the search goal (topic selection) to synthesizing the encountered information (search closure).
Research Questions.Motivated by gaps observed in existing literature (Section 2) and focused on setting a research foundation on the impact of IH on search on debated topics with a specific focus on confirmation bias, we outline an initial set of research questions: RQ1 -RQ4 focus on effective boosting interventions and their impact on search behavior and opinion formation; the remaining guide a wider research scope encompassing personalization, long-term effects, other search tasks, and alternative search paradigms.
(RQ1) What are effective interventions to boost IH in web searchers?(RQ2) Does boosting IH modify users' search behavior when searching for resources on debated topics?(RQ3) Does boosting IH mitigate confirmation bias during search on debated topics?(RQ4) Does boosting IH enable better-informed opinion-forming and decision-making in users?(RQ5) How would the boosting interventions need to be personalized to be effective for users with different characteristics and abilities?(RQ6) What are the longer-term effects of boosting IH on search behavior and search consequences?(RQ7) How would the boosting interventions need to be adapted to support better search behavior and consequences for search tasks beyond debated topics?(RQ8) How would the boosting interventions need to be adapted to be effective in non-textual search paradigms (e.g. with conversational agents or chatbots)?
Challenges.Certain challenges will be encountered when planning how to evaluate the applicability and effectiveness of boosting approaches against confirmation bias during web search for resources on debated topics.
Categorizing Search Behavior.We are interested in the effects of boosting interventions on the quality of users' overall search behavior throughout the ISP.For this, we need to operationalize and map behavioral variables (queries, clicks, time spent, etc.) to categorise search behavior (e.g., exploratory vs. lookup, viewpoint biased vs. unbiased, focused vs. unfocused, or motivated vs. unmotivated).This mapping can be informed by prior attempts, for example, identifying behavioral indicators of exploratory vs. lookup search behavior [2], or mapping quantitative indicators of search behavior to different search roles that users take on [22,28].It would also be interesting to look at the general relationship between IH and search roles by investigating whether IH affects people differently who tend to take on different search roles.
Measuring IH.To test the degree to which interventions lead to increased IH, we need to devise a method to assess the change in the participants' IH from prior-to post-boosting.However, applying a questionnaire might be boosting IH itself and thus distort the results.Alternatively, the effect of interventions on IH could be investigated in laboratory studies to capture behavioral metrics, e.g., with NLP-or eye-tracking-based measures of IH [4,9].
Evaluating Search Consequences.Ultimately, we aim for simple and applicable interventions that positively impact the consequences of web searches on debated topics, namely by achieving better-informed attitude-forming and decision-making.While informedness can be measured with knowledge questionnaires on the topic, the responsiveness of a decision or attitude to the information someone has is very challenging.This could be evaluated with an artificial topic in a very controlled setting with a designate "correct" attitude participants should have after the search sessions.However, such a setting would likely fail to reflect the complexity of search on debated topics and thus lack ecological validity.
Creating Realistic Search Scenarios.To ensure the validity of potential findings, the search scenario and environment need to assimilate real-world search on debated topics.This requires creating a sound task initiation, recreating a search environment that initiates confirmation bias and allows for a large range of possible search interactions, while having access to information about the search results' viewpoints to evaluate bias.

INITIAL RESEARCH STEPS
In this section, we describe the first steps on how we plan to advance knowledge toward building a foundation for our vision.We describe the boosting interventions that we suggest to explore and the stage of the ISP they target.In addition, we outline the design of the user studies we will conduct in order to investigate the impact of the proposed interventions.
Boosting Interventions.The interventions that we plan to investigate are motivated by the assumption that most individuals place a high value on principles that oppose confirmation bias and align with high levels of IH (such as objectivity, open-mindedness, tolerance, and curiosity).Thus, the interventions will primarily aim at (1) priming IH by explaining the benefits of high IH in terms of societal values, such as open-mindedness and tolerance for diverse perspectives, (2) reminding searchers of their own values by asking them to fill in a questionnaire, that measures trait-level IH, and (3) reinforcing their own values by combining both approaches to trigger and exploit participants' social desirability bias (see Figure 3).Social desirability bias would lead to searchers reporting higher levels of IH than they exhibit in their usual behavior.This might reinforce users' values and consequently motivate users to behave accordingly, thus with increased IH, during the search task.For the remind and reinforce interventions, we will ask users to fill out the extensively validated IH questionnaire developed by Alfano et al. [1].This is a high-dimensionality questionnaire that permits reminding searchers of the various values related to IH.
Boosting Goal.We anticipate that effective boosting intervenwould positively impact users' search behavior throughout the ISP (see Figure 2).To control scope, in the first iteration of our work, we will focus on the pre-focus exploration and focus formation (queries) and information collection (clicks) stages of the ISP.We hypothesize that the boosting interventions increase the diversity of viewpoints users' engage with and their overall exploration of search results.
Planned Method.We are currently designing two user studies to examine the impact of varying degrees of specificity in a boosting approach on participants' metacognitive state-level IH (RQ1 in § 3) and participants' search behavior (RQ2 and RQ3 in § 3).The design of these studies is inspired by the experimental design of Lorenz-Spreen et al. [31] who investigated the effect of boosting interventions with varying complexity on participants' accuracy of identifying micro-targeted ads.With these studies, we aim to identify effective interventions with low complexity to permit applicability in a real-world web search setting.Both studies will be conducted in form of randomized controlled trials between-subjects designs.We plan to manipulate the independent variable of what boosting intervention participants will be exposed to (none, prime, remind, reinforce) (see Figure 3), and measure effects on their state-level of IH (study 1), and search behavior (study 2).To measure changes in participants' state-level of IH from prior-to post-boosting, we plan to apply state-level IH questionnaires that ask users to report on their IH in the specific context of the task.In study 2, we will only include interventions that effectively boosted IH in study 1.If none of the suggested interventions proves effective, we will explore whether certain user traits have an impact on the effectiveness of the interventions, and devise and test alternative, if required, personalized interventions to boost IH.For this second study, we plan to log participants' search behavior (e.g., queries: number, length; clicks: number, dwell time, lowest rank, diversity of viewpoints) to capture potential effects of the interventions on participants' overall search behavior and their confirmation bias.We will preregister the design of the studies, material, data collection procedure, variables, hypotheses, planned sample, and planned analyses prior to collecting any data.We plan to recruit participants via online participant pools, such as Prolific [42].

CONCLUSION
With the work we presented in this paper, we call for a shift in how we attempt to tackle the manifold and complex challenges inherent to search on debated topics.In their majority, approaches so far have focused on directly nudging users' behavior.Instead, we propose to modify user behavior indirectly by boosting users' metacognitive state of intellectual humility.Doing so would avoid the risk of harming user autonomy and address the broader issues related to search on controversial topics in a more comprehensive and sustainable manner.
To make a meaningful impact, it is crucial to take a holistic approach to problem-solving, driven by interdisciplinary efforts from information retrieval and human-computer interaction researchers, social scientists, cognitive psychologists, philosophers, and legislators, to name a few.Together, researchers and (industry) practitioners need to identify efficient boosting approaches, develop the technical aspects of integrating boosting interventions into search engines, design an easy-to-use interface that responds to different users, evaluate the individual and societal impact, study the ethical implications and redefine the responsibilities of search engines, and understand and advance the regulatory context of such responsibilities and interventions.Ultimately, this work aims to contribute to a more well-informed and inclusive society where search empowers individuals to form opinions and make decisions based on a well-rounded understanding of debated topics.

Figure 1 :
Figure 1: Search engine result pages (SERPs) generated by Google on Jan 18 2023 for queries on debated topics.Although SERP results are relevant to the respective queries, it is evident that query variations lead to different results: In the SERP for the query should I go vegan?(left), most results support veganism; for is veganism dangerous?(right) all results oppose veganism.

Figure 2 :
Figure 2: Boosting and nudging for well-informed opinions and decisions after searching on debated topics.Boosting approaches that target metacognitive states could increase users' intellectual humility by fostering their existing competencies and motivations.Boosting the state of IH could indirectly modify users' search behavior (e.g., increased exploration of diverse viewpoints) and search consequences (e.g., opinions based on more accurate evaluation of strength of arguments), differing from the direct manipulation of users' search behavior of existing nudging approaches.

Figure 3 :
Figure 3: Planned boosting approaches; we will test the interventions in a between-subjects design.Prime: Users are primed with an explanation of the societal benefits of high IH.Remind: Users are reminded of their values by asking them to reflect on their trait of IH with a questionnaire.Reinforce: Users are primed and then asked to reflect on their trait of IH which might reinforce their values towards higher IH.