Synthetic Cognitive Feedback: Knowledge Erosion by Recursive Training of AI Generative Models

Synthetic Cognitive feedback is a recursive process in which a generative artificial intelligence (AI) model is trained on data it has produced itself. This loop can amplify internal biases, degrade output quality, and detach models from real-world data. As human-generated data becomes scarcer, such systems could increasingly rely on synthetic information, leading to possible scenarios where models are trained solely on outputs of other models. To reflect on this phenomenon, we present a musical piece in which, while a human performer plays, an AI is trained in real time on the performer’s past actions and recursively retrained on its own outputs. As the composition unfolds, the model gradually overrides human control and eventually takes full command of the execution. The work highlights the risks of over-relying on AI while neglecting the development of human knowledge, and encourages reflection on the shifting balance between authorship, originality, and machine-driven creation.