The Impact of Incentives on Survey Response Rates in the NSF ATE Community

Evaluation surveys are widely used by projects within the National Science Foundation’s Advanced Technological Education (ATE) program to understand the reach, effectiveness, and impact of project activities. As a result, project staff and evaluators often aim to maximize survey response rates, since higher rates can improve data quality, representativeness, and the usefulness of findings for learning and decision-making. This study examined whether incentives were associated with higher response rates in a web-based survey involving members of the ATE community. Using a stratified randomized experimental design, 2,943 individuals were assigned to one of three conditions: a guaranteed $5 incentive, entry into an iPad lottery, or no incentive. Analyses examined differences by incentive condition, ATE affiliation, frequency of EvaluATE use, and respondent role. Overall, participants who were offered an incentive were significantly more likely to complete the survey than those who were not (30% vs. 21%; p < .01). Incentives were associated with significantly higher response rates among non-ATE participants (27% vs. 17%; p < .01), as well as among both more-frequent users of EvaluATE resources (39% vs. 26%; p < .01) and less-frequent users (26% vs. 19%; p < .05). Differences among ATE participants overall and ATE evaluators were not statistically significant. Together, these findings provide practical, evidence-informed guidance for evaluators and project staff designing survey strategies to increase participation, broaden representation across groups, and