Transparency and replicability in mixed-methods designs using experiments

The use of mixed methods designs containing experiments has become more popular in the social sciences over the past decades (Harbers and Ingram 2020; Seawright 2016; Weller and Barnes 2014). In the analysis of experimental results, the qualitative component is typically used to illuminate causal mechanisms (Dunning 2015; Paluck 2010). However, when it comes to improving experimental designs, the capacity of qualitative methods to improve measurement is discussed less frequently. Prior to the analysis of data, qualitative methods can be used to design better contextualized, more realistic, experimental treatments (Dunning 2008; Dunning and Harrison 2010; Seawright 2016; 2021)3. Yet, the process of using qualitative methods to improve treatment design, for example, through the establishment of a sequence that can be replicated, is rarely formalized. We highlight the importance of standardizing the use of qualitative research to improve experimental treatments by pre-registering it as part of a pre-analysis plan (hereafter, PAP). In formalizing this process, researchers can contribute to the transparency and replicability of the entire research process.