Heteromation in Citizen Science: The Division of Labor Between Citizens, Experts, and Machines

Citizen science is a promising field for the creation of human-machine systems with increasing computational abilities, as several projects generate large datasets that can be used as training materials for machine learning models. This paper aims to identify the forms of human-machine learning integration in citizen science projects. The fifty articles examined in this systematic review report on projects combining human and machine efforts for analyzing, coding, classifying, and clustering data provided, for example, by cameras and telescope images. Machine learning is used at various stages of the data life cycle, through algorithms that perform tasks like classification, regression, clustering, and association. The findings highlight the character of the projects as heteromated systems, wherein human participation remains crucial and volunteer and ML efforts are often positioned as complementary rather than mutually exclusive. While leveraging the complementarity of strengths is one of the main arguments to combine humans and machines and enhance their respective capabilities, essentializing the attributes of humans and machines should be avoided. Treating these attributes as stable and natural does not take into account that cognitive work will be shifting between humans and machines, as the list of research tasks that machines can do is growing, although algorithms are still second to humans on recognizing patterns and they have longer learning curves. The findings can help researchers and practitioners to better understand human-machine integration in citizen science and point to unexplored areas. The emerging academic field of collective intelligence, increasingly interested in combining human intelligence with AI, can also find the review relevant.