10.5281/zenodo.4468277
https://zenodo.org/records/4468277
oai:zenodo.org:4468277
Moreo, Alejandro
Alejandro
Moreo
0000-0002-0377-1025
Italian National Council of Research
Sebastiani, Fabrizio
Fabrizio
Sebastiani
0000-0003-4221-6427
Italian National Council of Research
Re-Assessing the "Classify and Count" Quantification Method
Zenodo
2021
2021-03-28
eng
10.5281/zenodo.4468276
https://zenodo.org/communities/ai4media
Creative Commons Attribution 4.0 International
Learning to quantify (a.k.a. quantification) is a task concerned with training unbiased estimators of class prevalence via supervised learning. This task originated with the observation that “Classify and Count” (CC), the trivial method of obtaining class prevalence estimates, is often a biased estimator, and thus delivers suboptimal quantification accuracy. Fol- lowing this observation, several methods for learning to quantify have been proposed and have been shown to outperform CC. In this work we contend that previous works have failed to use properly optimised versions of CC. We thus reassess the real merits of CC and its variants, and argue that, while still inferior to some cutting-edge methods, they deliver near-state-of-the- art accuracy once (a) hyperparameter optimisation is performed, and (b) this optimisation is performed by using a truly quantification-oriented evaluation protocol. Experiments on three publicly available binary sentiment classification datasets support these conclusions.