Getting Perspectives on Quality in the Age of AI

Research Software Engineering (RSEng) is grounded in the belief that better, higher-quality software enables better research. As we look to the future of RSEng, we see that generative AI could substantially accelerate and expand the scope of research software development, but it also fundamentally shifts the nature of research software development by making code easier to generate than to understand, validate, or maintain. This is especially consequential in scientific computing, where software quality is directly tied to reproducibility, trust, and scientific validity. For RSEs, there is now an urgent need to advance a practical vision for how we will succeed in producing high-quality, trustworthy software under these new circumstances. 

We argue that in the age of AI for science, developers play an essential role in creating quality by reasoning about, negotiating, and ensuring it across the software lifecycle. We advance a theory that helps explain what, exactly, developers contribute by focusing on three constructs: values, which shape which quality attributes matter; expertise, which enables individuals to pursue and evaluate those attributes; and collaboration, which coordinates diverse forms of knowledge and accountability. To both build better AI tools for research software development and prepare our workforce, we need to better understand which values, what kinds of expertise, and what forms of collaboration will prove most helpful. For that, we seek expert input from Research Software Engineers(RSEs).

This poster provides participants with an entertaining, interactive experience with the core ideas of our research. As well as sharing the preliminary thoughts and opinions gathered from an interview study that is currently underway. This interview study was motivated by the work this poster is highlighting. By showing the preliminary results we hope to initiate a dialog with RSEs. As such invite the audience to reflect on what quality means in their own practice, and the work that is needed to achieve it in concert with AI. Attendees can contribute responses on post-it notes attached to the poster and will be invited to continue the conversation through a survey. We will encourage those who wish to learn more about our team’s ongoing work on these topics to attend our talk "We Create Quality: Towards a Human-Centric Theory of Research Software Quality in the Age of AI”.