Compositional Multi-Agent Learning with Colored Markov Polycategories

Formalizing the structure and learnable dynamics of evolving multi-agent, heterogeneous systems demands a compositional framework. This work presents the co-indexed colored Markov polycategory (CCMP), a novel categorical framework designed to address this challenge. The CCMP unifies polycategorical structures for agent interactions, Markovian semantics for stochastic processes, type theory via coloring for agent heterogeneity, and co-indexed categories for describing system dynamics, thereby providing a language for these systems. A methodology for multi-agent learning, employing gradient-based learning through adjoint sensitivity propagation, is developed within the CCMP, enabling system parameters to be learned from data. This approach facilitates the compositional design and probabilistic learning of dynamic multi-agent systems.