Loss of Execution Truth in Layered Cloud Orchestration: A Case Study of OpenStack and Kubernetes Under CPU Oversubscription

This report presents a technical case study of a production failure in a layered cloud environment where Kubernetes is deployed inside virtual machines managed by OpenStack. While such architectures are widely used for organizational, security, and operational reasons, they introduce multiple independent control planes competing for the same physical execution substrate.

The analyzed incident demonstrates how aggressive CPU oversubscription and cluster-wide saturation can lead to a loss of controllability: virtual machines remain logically ACTIVE while becoming physically non-responsive; live migration fails; and recovery requires forced hypervisor reboot. To capture this failure mode, the report introduces the concept of loss of execution truth, defined as a divergence between control-plane state and execution-plane reality.

The report develops a simple analytical model of layered oversubscription, showing how OpenStack-level and Kubernetes-level scheduling decisions multiply into an effective contention factor for physical CPU resources. This model provides a concise explanation of why orchestration and self-healing mechanisms fail precisely under the conditions in which they are most needed.

Beyond the specific incident, the report draws broader architectural and operational conclusions for modern cloud platforms, emphasizing the limits of layered orchestration, the need to govern overcommit as a reliability parameter, and the importance of execution-level understanding alongside formal operational processes.

This work is intended for cloud architects, site reliability engineers, and infrastructure researchers interested in the reliability and failure modes of large-scale virtualized and containerized platforms.