Some rough benchmarks for testing how long things take…
library(tidyverse)
## -- Attaching packages ------------------------------------------------------------ tidyverse 1.2.1 --
## v ggplot2 3.2.1 v purrr 0.3.2
## v tibble 2.1.3 v dplyr 0.8.3
## v tidyr 1.0.0 v stringr 1.4.0
## v readr 1.3.1 v forcats 0.4.0
## -- Conflicts --------------------------------------------------------------- tidyverse_conflicts() --
## x dplyr::filter() masks stats::filter()
## x dplyr::lag() masks stats::lag()library(tidybayes)
spread/gather_draws
k = 50000 set.seed(1234) df = data.frame( .chain = NA, .iteration = NA, .draw = 1:k, `a[2]` = rnorm(k), `b[2]` = rnorm(k), `c[2]` = rnorm(k), `d[2]` = rnorm(k), `e[2]` = rnorm(k), `f[2]` = rnorm(k), `g[2]` = rnorm(k), `h[2]` = rnorm(k), `i[2]` = rnorm(k), `j[2]` = rnorm(k), `k[2]` = rnorm(k), `l[2]` = rnorm(k), stringsAsFactors = FALSE, check.names = FALSE ) %>% as_tibble() df
## # A tibble: 50,000 x 15
## .chain .iteration .draw `a[2]` `b[2]` `c[2]` `d[2]` `e[2]` `f[2]`
## <lgl> <lgl> <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 NA NA 1 -1.21 0.491 -1.42 0.892 -0.553 -0.276
## 2 NA NA 2 0.277 0.0250 0.317 -0.573 -1.13 0.919
## 3 NA NA 3 1.08 1.30 0.728 0.0640 0.0360 -1.02
## 4 NA NA 4 -2.35 -0.235 -1.90 0.0434 -1.17 -1.42
## 5 NA NA 5 0.429 -0.453 -1.14 1.80 0.0443 -0.660
## 6 NA NA 6 0.506 -0.0111 0.577 -2.58 0.269 0.296
## 7 NA NA 7 -0.575 0.933 0.198 -0.218 -0.182 -0.906
## 8 NA NA 8 -0.547 -0.729 1.07 2.15 -1.27 -1.05
## 9 NA NA 9 -0.564 1.40 0.626 1.11 0.738 -0.778
## 10 NA NA 10 -0.890 -0.0584 -0.700 -0.956 -2.27 1.02
## # ... with 49,990 more rows, and 6 more variables: `g[2]` <dbl>,
## # `h[2]` <dbl>, `i[2]` <dbl>, `j[2]` <dbl>, `k[2]` <dbl>, `l[2]` <dbl>This should be fairly fast. Last test was ~= 0.03 seconds on monarch.
system.time(spread_draws(df, `.`[index], regex = TRUE))
This is definitely slower than it could be, likely due to use of joins with chain information involved. This could almost definitely be made faster if nesting was used all the way up the spread_draws pipeline and chain information added at the last possible moment. Last test was ~= 0.35 seconds on monarch.
system.time(spread_draws(df, a[index], b[index], c[index], d[index], e[index], f[index], g[index], h[index], i[index], j[index], k[index], l[index] ))
This is slower than the spread_draws version even though it shouldn’t really have to be. This is a result of spread_draws_long_ doing a spread that is later undone by gather_draws. Could either use a flag to skip this (although that is a little complicated as the | syntax might need the spread anyway) or rely on switching to nesting throughout the entire pipeline to just make the spread/gather round trip not so costly. Last test was ~= 0.13 seconds on monarch.
system.time(gather_draws(df, `.`[index], regex = TRUE))
This tends to be slower than spreading without array columns. Last test was ~0.08 seconds on monarch.
data(RankCorr, package = "ggdist") system.time(spread_draws(RankCorr, b[.,.]))
And this is currently very slow. Last test was ~0.5 seconds on monarch.
system.time(spread_draws(df, a[.]))