Performance Analysis of CI/CD Pipelines Using (Colored) Generalized Stochastic Petri Nets

DevOps is enjoying widespread adoption by enterprises. Many of

them are utilizing CI/CD pipelines. These allow them to define a

chain of processing, in which elements of the DevOps toolchain,

such as building, testing, and deploying, are executed. As software

can only reach users through this pipeline, its speed is of quintes-

sential concern, as delays or failures of the pipeline can cause an

abrupt halt to development and deployment. However, there is a

severe lack in methods to evaluate the speed of a pipeline. Thus,

we propose an approach to detect performance issues of pipelines

using Petri Nets. Additionally, we propose an approach to predict

to what degree pipeline execution must be adjusted, in order to

prevent performance issues entirely. Our approach utilizes StalkCD,

which allows the transformation of pipelines into BPMN and back.

We use the model it creates, and transform it into a Petri Net instead.

We then run a steady state simulation on the Petri Net in order to

get estimates on how the pipeline will perform under various loads.

As StalkCD currently only transforms Jenkins pipelines, the pat-

terns we use to create our Petri Net model were derived primarily

from Jenkins pipelines as well. Additionally, to obtain test-related

information, such as service rates of pipeline elements, we have de-

veloped scripts to transform data provided by various Jenkins APIs

into data models which are used by our transformation. To evaluate

the validity of our approach, we analyze the speed of simulation

of our Petri Net and how accurate our predictions, stemming from

the Petri Net simulation, are to data obtained from actual pipeline

executions. Our result is that the Petri net model we created can

not make accurate predictions of Jenkins pipelines. We discovered

this is due to patterns in Jenkins which we did not or could not

cover in our model. Additionally, we discovered our evaluation

could not produce results necessary to validate our approach. To

create an accurate model and evaluate its predictions would require

a massive amount of research and effort.