Analyzing storage access data with Apache-Spark and Jupiter notebooks

Running a data center is never a trivial job. In addition to daily routine tasks, service operation teams have to provide a meaningful information for monitoring, reporting and access pattern analytic. The dCache production instances at DESY, produce gigabytes of billing files per day. However, with a help of modern BigData analysis tools like Apache-Spark and Jupiter notebooks such task can be easily achieved. Moreover, the tool set for storage access analysts can be shared with scientific community making it re-usable computational resource as well as shared knowledge