Measuring costly concepts: validation samples for measuring many-n case

Costly concepts are concepts that are expensive or otherwise resource-intensive to obtain measurement for over many cases. Costly concepts are present across the social sciences, though particularly in the subnational study of comparative politics. Subnational democracy, local-level armed group presence, and municipal corruption are all costly concepts for which measurement requires fine-grained data that may be practically impossible to collect for many units where the data are not already available to researchers. In the absence of actual measures of costly concepts, scholars will often substitute measurement by using proxy variables in empirical analyses, which causes nonrandom measurement error where measurements of the costly concept and proxies are not identical. This nonrandom measurement error means we risk conducting biased analyses when we cannot overcome the structural challenges that preclude precise measurement of costly concepts.