Published November 4, 2020 | Version v1

Accounting for New Types of Resource Consumption in a Federated Cloud

  • 1. CESNET

Description

As infrastructures and cloud services evolve, resource consumption is more flexible and users are often allowed to reserve resources without actual consumption. Relevant standardization bodies have developed new types of accounting record specifications, and new or updated tools are required to keep track of resource usage. The reaction to this is the development of a new accounting tool – GOAT.

GOAT – GO Accounting Tool – is a service running in the background and waiting for a connection from a compatible client. The client connects to a cloud management framework, extracts computing data about projects, servers, networks, and storages, filters them accordingly, and sends them to a server for further processing. Multiple clients can use the server at once. When the server receives accounting data, it is transformed into the configured format and writes them to the destination file. The consumer collects data into a central accounting database where it is processed to generate statistical summaries.
For now, the GOAT project supports two cloud computing platforms on the client side – OpenNebula and Openstack. Thanks to its use of standard accounting record formats it can work with different consumers. The ones used in the real world are APEL and Prometheus.

This Demonstration shows how the GOAT client extracts accounting data from a cloud management platform, sends them to the GOAT server where they are transformed, and how Prometheus and Grafana process them and present various views of resource usage.

Notes

Additional funding: The Ministry of Education, Youth and Sports (CZ), project Large Infrastructures for Research, Experimental Development and Innovations "e-INFRA CZ– LM2018140".

Files

AccountingDemo.mp4

Files (234.7 MB)

Name Size
md5:10407b9f333a1fd2f364ff984ffdc1ee
234.7 MB Preview Download
md5:51b706b1a5028492f0cb2ee72c4ec619
16.1 kB Download

Additional details

Funding

European Commission
EOSC-hub - Integrating and managing services for the European Open Science Cloud 777536