Genomic-Dependent Radiochemotherapy Optimization in Structured Tumor Dynamics: Existence, Stability, and Optimal Control Analysis

Cancer treatment response varies substantially across patients due to genomic heterogeneity, particularly differences in tumor radiosensitivity. Classical tumor growth models typically assume homogeneous cell populations and fixed treatment parameters, which limits their applicability to personalized therapy design.

In this work, we develop a nonlinear structured population model describing tumor dynamics with genomic-dependent radiosen- sitivity. The tumor population is structured by a continuous radiosensitivity trait and evolves according to a nonlinear integro– differential equation incorporating radiotherapy and chemotherapy effects, phenotypic mutation, and density-dependent competi- tion for resources. The resulting system is formulated in an infinite-dimensional framework.We establish well-posedness of the model by proving existence, uniqueness, positivity, and boundedness of solutions using semigroup and fixed-point arguments. An optimal control problem is formulated to determine personalized radiotherapy and chemotherapy protocols that minimize tumor burden while penalizing treatment toxicity. Using techniques from functional analysis and infinite-dimensional optimal control theory, we prove existence of optimal controls and derive first-order necessary optimality conditions via Pontryagin’s Maximum Principle.

A stability analysis of equilibria is performed, and the model reveals an evolutionary selection mechanism through which treat- ment pressure induces shifts in the genomic trait distribution toward resistant tumor phenotypes. These results provide a rigorous mathematical framework for personalized radiochemotherapy optimization and contribute to the theory of structured population dynamics in mathematical oncology.