Measures of divergence for binary data used in biodistance studies

Biodistance analysis can elucidate various aspects of past population structure. The most commonly adopted measure of divergence when estimating biodistances is the mean measure of divergence (MMD). The MMD is an unbiased estimator of population divergence but this property is lost when the dataset includes variables with very high or low frequency. In the present paper, we examine new measures of divergence based on untransformed binary data and the logit and probit transformations. It is shown that a measure of divergence based on untransformed data is a better unbiased estimator of population divergence. The conventional MMD is a satisfactory distance measure for binary data; however, it may produce biased estimations of population divergence when there are many traits with frequencies lower than 0.1 or/and greater than 0.9. Finally, the measures of divergence based on the probit and logit transformations are usually biased estimators.