Working paper Open Access

A Formal Semantics for Data Analytics Pipelines

Drocco, Maurizio; Misale, Claudia; Tremblay, Guy; Aldinucci, Marco


JSON Export

{
  "conceptrecid": "799186", 
  "created": "2017-05-05T13:10:00.694406+00:00", 
  "doi": "10.5281/zenodo.571802", 
  "files": [
    {
      "bucket": "6125b8f8-f0dd-4574-a74c-e73d81b1cfc9", 
      "checksum": "md5:a6859d2c8bd970e6cf5504ffca5ca393", 
      "key": "1705.01629.pdf", 
      "links": {
        "self": "https://zenodo.org/api/files/6125b8f8-f0dd-4574-a74c-e73d81b1cfc9/1705.01629.pdf"
      }, 
      "size": 258335, 
      "type": "pdf"
    }
  ], 
  "id": 571802, 
  "links": {
    "badge": "https://zenodo.org/badge/doi/10.5281/zenodo.571802.svg", 
    "bucket": "https://zenodo.org/api/files/6125b8f8-f0dd-4574-a74c-e73d81b1cfc9", 
    "doi": "https://doi.org/10.5281/zenodo.571802", 
    "html": "https://zenodo.org/record/571802", 
    "latest": "https://zenodo.org/api/records/571802", 
    "latest_html": "https://zenodo.org/record/571802"
  }, 
  "metadata": {
    "access_right": "open", 
    "access_right_category": "success", 
    "creators": [
      {
        "affiliation": "Department of Computer Science, University of Torino, Italy", 
        "name": "Drocco, Maurizio"
      }, 
      {
        "affiliation": "Department of Computer Science, University of Torino, Italy", 
        "name": "Misale, Claudia"
      }, 
      {
        "affiliation": "D\u00e9pt. d'Informatique, Universit\u00e9 du Qu\u00e9bec \u00e0 Montr\u00e9al, Canada", 
        "name": "Tremblay, Guy"
      }, 
      {
        "affiliation": "Department of Computer Science, University of Torino, Italy", 
        "name": "Aldinucci, Marco"
      }
    ], 
    "description": "<p>In this report, we present a new programming model based on Pipelines and Operators, which are the building blocks of programs written in PiCo, a DSL for Data Analytics Pipelines. In the model we propose, we use the term Pipeline to denote a workflow that processes data collections -- rather than a computational process -- as is common in the data processing community. The novelty with respect to other frameworks is that all PiCo operators are polymorphic with respect to data types. This makes it possible to 1) re-use the same algorithms and pipelines on different data models (e.g., streams, lists, sets, etc); 2) reuse the same operators in different contexts, and 3) update operators without affecting the calling context, i.e., the previous and following stages in the pipeline. Notice that in other mainstream frameworks, such as Spark, the update of a pipeline by changing a transformation with another is not necessarily trivial, since it may require the development of an input and output proxy to adapt the new transformation for the calling context. In the same line, we provide a formal framework (i.e., typing and semantics) that characterizes programs from the perspective of how they transform the data structures they process -- rather than the computational processes they represent. This approach allows to reason about programs at an abstract level, without taking into account any aspect from the underlying execution model or implementation.</p>", 
    "doi": "10.5281/zenodo.571802", 
    "grants": [
      {
        "acronym": "TOREADOR", 
        "code": "688797", 
        "funder": {
          "acronyms": [
            "EC"
          ], 
          "doi": "10.13039/501100000780", 
          "links": {
            "self": "https://zenodo.org/api/funders/10.13039/501100000780"
          }, 
          "name": "European Commission"
        }, 
        "links": {
          "self": "https://zenodo.org/api/grants/10.13039/501100000780::688797"
        }, 
        "program": "H2020", 
        "title": "TrustwOrthy model-awaRE Analytics Data platfORm"
      }, 
      {
        "acronym": "RePhrase", 
        "code": "644235", 
        "funder": {
          "acronyms": [
            "EC"
          ], 
          "doi": "10.13039/501100000780", 
          "links": {
            "self": "https://zenodo.org/api/funders/10.13039/501100000780"
          }, 
          "name": "European Commission"
        }, 
        "links": {
          "self": "https://zenodo.org/api/grants/10.13039/501100000780::644235"
        }, 
        "program": "H2020", 
        "title": "REfactoring Parallel Heterogeneous Resource-Aware Applications  - a Software Engineering Approach"
      }
    ], 
    "keywords": [
      "Big Data analytics", 
      "Types", 
      "Parallel computing", 
      "Distributed computing"
    ], 
    "license": {
      "id": "CC-BY-4.0"
    }, 
    "notes": "https://arxiv.org/abs/1705.01629", 
    "publication_date": "2017-05-03", 
    "relations": {
      "version": [
        {
          "count": 1, 
          "index": 0, 
          "is_last": true, 
          "last_child": {
            "pid_type": "recid", 
            "pid_value": "571802"
          }, 
          "parent": {
            "pid_type": "recid", 
            "pid_value": "799186"
          }
        }
      ]
    }, 
    "resource_type": {
      "subtype": "workingpaper", 
      "title": "Working paper", 
      "type": "publication"
    }, 
    "title": "A Formal Semantics for Data Analytics Pipelines"
  }, 
  "owners": [
    28899
  ], 
  "revision": 5, 
  "stats": {
    "downloads": 8.0, 
    "unique_downloads": 8.0, 
    "unique_views": 98.0, 
    "version_downloads": 8.0, 
    "version_unique_downloads": 8.0, 
    "version_unique_views": 98.0, 
    "version_views": 98.0, 
    "version_volume": 2066680.0, 
    "views": 98.0, 
    "volume": 2066680.0
  }, 
  "updated": "2017-09-08T07:53:19.933628+00:00"
}
98
8
views
downloads
All versions This version
Views 9898
Downloads 88
Data volume 2.1 MB2.1 MB
Unique views 9898
Unique downloads 88

Share

Cite as