Design and implementation of a web survey on the effects of evaluation on academic research

The project ‘The effects of evaluation on academic research: knowledge production and methodological issues’ (PRIN–Research Projects of National Interest 2017 - Prot. 2017NKWYFC), coordinated by CNR-IRCrES, aims to investigate the effects of the evaluation implemented by the government on research work and knowledge production, taking as a case the academics working in Italian universities. The project adopts an interdisciplinary approach that combines different skills in social sciences (sociology, economics and organisational studies), and qualitative and quantitative methodological tools. As regards the latter, as part of the activities of the CNR-IRCrES, a web survey has been launched between October 2020 and March 2021 at national level to collect information on the effects of the evaluation on various dimensions related to the academic research, including the decisions on the contents of research activities, the modes of codification of knowledge produced, the research networking, etc. The target population has been made up of all academics from the universities subject to evaluation by the Italian National Agency for the Evaluation of Universities and Research Institutes (ANVUR). The invitation link to fill in the online questionnaire has been sent to a probability sample selected through a ‘multi-stage’ stratified sampling method. The survey received 1,365 responses, yielding a response rate
slightly higher than the 43%. This contribution outlines the survey's methodological approach, presenting the various stages of the research design. In addition, an analysis of the response rate and the sample representativeness in relation to the reference population will be presented. The design and the stages of implementation of the survey demonstrate their value as an original and
robust methodological proposal for studying the research evaluation effects at the micro-level of the producers of knowledge.