Population size differences can lead to biases in phylogenetic inference and introgression detection in the presence of purifying selection

Phylogenetic reconstruction and introgression detection are based on assumptions about the probability distribution of gene tree topologies. Initial evidence has suggested that population size differences can affect the probability distribution of gene tree topologies in the presence of purifying selection. Nevertheless, the impact of this phenomenon on phylogenetic reconstruction and introgression detection remains to be explored. Additionally, the mechanism underlying this phenomenon remains elusive. Here, using the population genetic simulator SLiM, we provide evidence that in the presence of purifying selection, population size differences can cause biases in phylogenetic inference. We also provide evidence that in the presence of purifying selection, population size differences can cause statistics used for introgression detection to exhibit patterns resembling those caused by introgression. Additionally, a theoretical analysis is presented to show that under purifying selection, the way in which single-population genealogies are connected together to form a gene tree can differ from that under neutral evolution as it is affected by population size differences. Consequently, the probability distribution of gene tree topologies under purifying selection is not identical to that under neutral evolution but instead is affected by population size differences. This work underscores the importance of considering the potential confounding impact of purifying selection on phylogenetic inference and introgression detection.