New sufficiency and necessity measures for model building with Coincidence Analysis

Coincidence Analysis (CNA) is a configurational comparative method of causal learning that has seen a significant uptick in applications in public health in recent years. To build its causal models, CNA searches for redundancy-free relations of sufficiency and necessity in data using a sufficiency measure called consistency and a necessity measure called coverage. This paper argues that consistency and coverage have severe limitations. In particular, they are not reliable when the relative frequencies of candidate causes and outcomes are at high or low extremes. We propose alternative sufficiency and necessity measures that are not affected by these limitations and benchmark them against standard consistency and coverage in an extended simulation experiment analyzing binary, so-called crisp-set, data. Across a wide range of data scenarios, the overall quality of CNA models built by means of the new measures is more than 20% higher than when models are built using the standard measures. Correspondingly, we recommend that the new measures are made available in relevant CNA software and that CNA users transition to building crisp-set models with them.