November 17, 2021 Presentation Open Access

Cherian, Deepak

Citation Style Language JSON Export

{
  "publisher": "Zenodo", 
  "DOI": "10.5281/zenodo.5772165", 
  "author": [
    {
      "family": "Cherian, Deepak"
    }
  ], 
  "issued": {
    "date-parts": [
      [
        2021, 
        11, 
        17
      ]
    ]
  }, 
  "abstract": "<p>The &quot;groupby&quot; or the &quot;split-apply-combine&quot; paradigm is ubiquitous in scientific analysis, though it may be named differently e.g. &quot;binning&quot;, &quot;histogramming&quot;, &quot;resampling&quot;, &quot;compositing&quot;, or &quot;climatology reductions&quot;. Xarray implements the groupby paradigm through a &quot;GroupBy&quot; object. Historically the underlying algorithm is not dask-aware, and tends to fail disastrously with large Pangeo-scale distributed workflows.&nbsp;Here I present &quot;flox&quot;: a new package that explores effective strategies for groupby reductions at scale with dask. Ongoing work will plug this package in to xarray in a backwards-compatible manner, allowing the community to seamlessly benefit from significantly more efficient groupby computations.See&nbsp;https://flox.readthedocs.io&nbsp;for more.</p>", 
  "title": "flox: Fast & furious GroupBy reductions with Dask at Pangeo-scale", 
  "type": "speech", 
  "id": "5772165"
}