Developing Adaptive Calibration and Benchmarking Methods for 6G Performance Measurement - Implementation and Conclusions from 6G-SANDBOX

The flexibility inherent to the 6G space through its use of virtualized/Service Based Architecture (SBA) networks means that generated performance results can be more difficult to contextualize than previous generations. In this space, experimentation, either with the purpose of evaluating the appropriate integration of a new network device, system or software or targeting the assurance of application-demanding performance service levels, needs to be done in such a way to ensure that the assessment is performed uninfluenced by the actual dimensioning constraints of the underlying beyond 5G (B5G)/6G experimentation platform. This paper explores the development of an Adaptive Calibration methodology, which can be used to provide context to results in the 6G space so that to ensure platform agnosticism. Exemplary results of its application in the 6G-SANDBOX project, exploiting four (4) different and geographically dispersed experimentation platforms, are presented hereby. The
methodology involves the establishment of a baseline environment which can be measured before the introduction of a device/system/software under test (SUT). The results of the baseline measurement can be compared to the results including the SUT to produce deltas in multiple KPIs. These deltas can then be used to provide context to results generated within a single platform or be used to compare multiple platforms. The use of this methodology within 6G-SANDBOX uncovered significant performance disparities between the partaking platforms related to virtualization methods, and it is currently being used to drive the platforms closer to parity, investigate the virtualization issues and quantify the performance of various components within each individual platform. In this regard, the application of the proposed methodology is proven substantial to increase confidence in the SUT assessment results performed over B5G/6G experimentation platforms.