The Cloud-based Workflow Manager (CloWM) - An integrated platform for highly scalable workflow execution

Data analysis in life sciences often suffers from reproducibility, standardization, and

accessibility issues due to the reliance on custom CLI scripts and the lack of easily

accessible compute resources. The Cloud-based Workflow Manager (CloWM)

addresses these challenges by providing a user-friendly platform for executing

complex data analysis workflows in a standardized and reproducible manner. The

system leverages cloud-based infrastructure to offer highly-scalable compute

resources and personalized data storage through sharable S3 buckets. By

encapsulating versioned Nextflow workflows as web services, developers can easily

share and execute their analysis pipelines. Moreover, to facilitate efficient data

management, workflow developers can upload and manage persistent and versioned

workflow resources - like large, static databases - in CloWM. These are then

accessible from within workflow executions, thus eliminating redundant data transfers

and reducing workflow execution time. A robust authentication and authorization

system, integrated with the LifeScience AAI SSO solution, ensures secure

personalized access and data protection. An intuitive web interface which supports

the automatic generation of workflow parameter interfaces, streamlines workflow

management and execution. In addition, a REST API secured by API tokens allows

programmatic interaction with the system, enhancing its flexibility and integration

capabilities. CloWM empowers researchers to focus on scientific insights by handling

and hiding the complexities of cloud-based infrastructure provisioning and distributed

data management while workflow developers benefit from a broader target audience

that can easily execute their workflows in a highly-scalable fashion. The CloWM

system is available under https://clowm.bi.denbi.de and the code can be found at

https://gitlab.ub.uni-bielefeld.de/cmg/clowm.