Improving Research Software through Short Feedback Loops

Research software engineering is often as volatile and resource-constrained as the research it supports. For instance, requirements are often difficult to define in advance, as the research team seeks to understand the problem domain through the research software. Further, the general direction of a project may shift with the changing focus of every new research project. Similarly, the code base might change unsteadily with every new project or even through ad hoc modifications to make the software suitable for a publication or presentation.

To ensure that we continuously add value to the research software in such a context, we need to understand and react quickly to changes in the requirements and the software behavior. In general software development practices, this is achieved through short feedback loops. Short feedback loops have been shown to be beneficial at various levels. For instance, quick feedback on the dynamic behavior of programs allows programmers to understand their software better and repair defects faster, automatic feedback through CI ensures that our software adheres to our expectations in different environments, and fast user feedback reduces wasted effort, as it ensures that newly introduced features do indeed add value for users.

Due to the volatile nature of research software engineering, getting feedback quickly is even more essential in research software engineering than in conventional software development. Through quick feedback, we can continuously update our understanding of the actual behavior of the research software and continuously re-align our understanding of what the research team needs, and make sure that our next changes are adding value. This leaves us with the challenge of determining which feedback is relevant to us and how to get it more quickly.

In this talk, we will take a look at the general feedback loops involved in research software engineering, how shortening these loops can be valuable, and several general principles and concrete techniques to shorten these loops at different levels of granularity in research software projects. We will draw inspiration from several communities whose ways of shortening feedback loops can be of interest to research software projects, such as live programming mechanisms, artistic coding performances, exploratory programming approaches, DevOps techniques, and design methodologies.