Software Open Access
Dr Rob Lyon
CSP to SDP NIP Data Rates & Data Models (version 1.1)
this release contains an ipython notebook and collection of python scripts. These make it easier to model SKA Science Data Processor (SDP) data rates. Diagrams are included that define the conceptual and logical structure of the Non-Imaging Processing (NIP) data models. Also included are activity diagrams for all NIP pipelines. Finally, formulas are presented that provide
accurate estimates of NIP pipeline data rates.
Author: Rob Lyon
Email : firstname.lastname@example.org
web : www.scienceguyrob.com
This release consists of,
The notebook describes the conceptual and logical data models for the Non-Imaging Processing (NIP) components of the SKA Science Data Processor (SDP), according to Software Engineering Institute (SEI) standards. The notebook was created for JIRA TSK-1294, which requested SEI compliant versions of the NIP data models. This document represents the 4th iteration (at least) of the data models work, however only the last two iterations, TSK-12 and TSK-73 are recorded in JIRA.
All data models presented in this notebook are described via Entity-Relationship Models (ERMs). These are sometimes referred to as an entity-relationship diagrams (ERDs). There is no standard notation for ERMs, however we have attempted to adhere to defacto standards. Unified Modelling Language (UML) activity diagrams are also used to characterise the processing activity within NIP, which ingests and outputs the data models.
This ipython notebook also models the input data rates and data volumes for NIP. It includes models for,
Data rate estimates are provided via interactive Python 2.7 code. This can be run either within the notebook, or externally via the supplied source code.
Given the use of ERM & UML, basic familiarity with SEI standards is recommended to fully understand the diagrams and terminology used in this notebook.
The code and the contents of this notebook are released under the GNU GENERAL PUBLIC LICENSE, Version 3, 29 June 2007. We kindly request that if you make use of the notebook, you cite the work appropriately.
The notebook builds upon earlier work by Dr. Lina Levin-Preston & Prof. Ben Stappers. Lina wrote the original data models document, did all the data volume/rate calculations, and provided the original notation! Ben worked on this too. This notebook relies greatly on their earlier work. Thank you Andrea Possenti, for providing valuable feedback which has helped produce the latest version.
Version 1.1 - Incorporates feedback from Andrea Possenti. Altered the description of a program and schedule block in Section 2.