Introduction to Cache

Eliot J. B. McIntire

June 15 2018

Source: vignettes/Intro-to-Cache.Rmd

Reproducible workflow

As part of a reproducible workflow, caching of function calls, code chunks, and other elements of a project is a critical component. The objective of a reproducible workflow is is likely that an entire work flow from raw data to publication, decision support, report writing, presentation building etc., could be built and be reproducible anywhere, on any computer, operating system, with any starting conditions, on demand. The reproducible::Cache function is built to work with any R function.

Differences with other approaches

Cache uses 2 key the archivist functions saveToLocalRepo and loadFromLocalRepo, but does not use archivist::cache. Similar to archivist::cache, there is some reliance on digest::digest to determine whether the arguments are identical in subsequent iterations; however, it also uses fastdigest::fastdigest to make it substantially faster in many cases. It also but does many things that make standard caching with digest::digest don’t work reliably between systems. For these, the function .robustDigest is introduced to make caching transferable between systems. This is relevant for file paths, environments, parallel clusters, functions (which are contained within an environment), and many others (e.g., see ?.robustDigest for methods). Cache also adds important elements like automated tagging and the option to retrieve disk-cached values via stashed objects in memory using memoise::memoise. This means that running Cache 1, 2, and 3 times on the same function will get progressively faster. This can be extremely useful for web apps built with, say shiny.

Function-level caching

Any function can be cached using: Cache(FUN = functionName, ...).

This will be a slight change to a function call, such as: projectRaster(raster, crs = crs(newRaster)) to Cache(projectRaster, raster, crs = crs(newRaster)).

This is particularly useful for expensive operations.

library(raster)
## Loading required package: sp
library(reproducible)

tmpDir <- file.path(tempdir(), "reproducible_examples", "Cache")
checkPath(tmpDir, create = TRUE)
## [1] "/tmp/RtmpeO1poy/reproducible_examples/Cache"
ras <- raster(extent(0,1000,0,1000), vals = 1:1e6, res = 1)
crs(ras) <- "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84"

newCRS <- "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
# No Cache
system.time(map1 <- projectRaster(ras, crs = newCRS))
##    user  system elapsed 
##   3.109   0.240   3.352
# With Cache -- a little slower the first time because saving to disk
system.time(map1 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir,
                         notOlderThan = Sys.time()))
##    user  system elapsed 
##   3.084   0.228   3.433
# vastly faster the second time
system.time(map2 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading cached result from previous projectRaster call, adding to memoised copy
##    user  system elapsed 
##   0.176   0.004   0.179
# even faster the third time
system.time(map3 <- Cache(projectRaster, ras, crs = newCRS, cacheRepo = tmpDir))
##   loading memoised result from previous projectRaster call.
##    user  system elapsed 
##   0.052   0.008   0.060
all.equal(map1, map2) # TRUE
## [1] TRUE
all.equal(map1, map3) # TRUE
## [1] TRUE

Caching examples

Basic use

library(raster)
# magrittr, if loaded, gives an error below
  try(detach("package:magrittr", unload = TRUE), silent = TRUE)

try(clearCache(tmpDir), silent = TRUE) # just to make sure it is clear

ranNumsA <- Cache(rnorm, 10, 16, cacheRepo = tmpDir)

# All same
ranNumsB <- Cache(rnorm, 10, 16, cacheRepo = tmpDir) # recovers cached copy
##   loading cached result from previous rnorm call, adding to memoised copy
ranNumsC <- rnorm(10, 16) %>% Cache(cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous 'rnorm' pipe sequence call.
ranNumsD <- Cache(quote(rnorm(n = 10, 16)), cacheRepo = tmpDir) # recovers cached copy
##   loading memoised result from previous rnorm call.
# Any minor change makes it different
ranNumsE <- rnorm(10, 6) %>% Cache(cacheRepo = tmpDir) # different

Example 1: Basic cache use with tags

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

showCache(tmpDir, userTags = c("objectName"))
## Cache size:
##   Total (including Rasters): 20.8 Kb
##   Selected objects (not including Rasters): 338 bytes
##                             artifact         tagKey
##  1: f3be0cb1ac973869cb34739d2d3fbedd         format
##  2: f3be0cb1ac973869cb34739d2d3fbedd           name
##  3: f3be0cb1ac973869cb34739d2d3fbedd          class
##  4: f3be0cb1ac973869cb34739d2d3fbedd           date
##  5: f3be0cb1ac973869cb34739d2d3fbedd        cacheId
##  6: f3be0cb1ac973869cb34739d2d3fbedd     objectName
##  7: f3be0cb1ac973869cb34739d2d3fbedd       function
##  8: f3be0cb1ac973869cb34739d2d3fbedd    object.size
##  9: f3be0cb1ac973869cb34739d2d3fbedd       accessed
## 10: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 11: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 12: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 13: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 14: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 15: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 16: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 17: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 18: f3be0cb1ac973869cb34739d2d3fbedd      preDigest
## 19: f3be0cb1ac973869cb34739d2d3fbedd      preDigest
## 20: f86f822df16a2ae18b84e7877ebfe51f         format
## 21: f86f822df16a2ae18b84e7877ebfe51f           name
## 22: f86f822df16a2ae18b84e7877ebfe51f          class
## 23: f86f822df16a2ae18b84e7877ebfe51f           date
## 24: f86f822df16a2ae18b84e7877ebfe51f        cacheId
## 25: f86f822df16a2ae18b84e7877ebfe51f     objectName
## 26: f86f822df16a2ae18b84e7877ebfe51f       function
## 27: f86f822df16a2ae18b84e7877ebfe51f    object.size
## 28: f86f822df16a2ae18b84e7877ebfe51f       accessed
## 29: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 30: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 31: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 32: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 33: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 34: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 35: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 36: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 37: f86f822df16a2ae18b84e7877ebfe51f      preDigest
## 38: f86f822df16a2ae18b84e7877ebfe51f      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:56
##  2:                                 Cache 2018-06-15 10:20:56
##  3:                               numeric 2018-06-15 10:20:56
##  4:                   2018-06-15 10:20:56 2018-06-15 10:20:56
##  5:      85874f26b2e0c1ef689a7d379d275ebf 2018-06-15 10:20:56
##  6:                                     a 2018-06-15 10:20:56
##  7:                                 rnorm 2018-06-15 10:20:56
##  8:                                   688 2018-06-15 10:20:56
##  9:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 10:                   withCallingHandlers 2018-06-15 10:20:56
## 11:                               saveRDS 2018-06-15 10:20:56
## 12:                               do.call 2018-06-15 10:20:56
## 13:                          process_file 2018-06-15 10:20:56
## 14:                         process_group 2018-06-15 10:20:56
## 15:                   process_group.block 2018-06-15 10:20:56
## 16:                            call_block 2018-06-15 10:20:56
## 17:                            block_exec 2018-06-15 10:20:56
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:56
## 19: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:56
## 20:                                   rda 2018-06-15 10:20:56
## 21:                                 Cache 2018-06-15 10:20:56
## 22:                               numeric 2018-06-15 10:20:56
## 23:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 24:      e37bb635c97bc2eeecab63816b881bbc 2018-06-15 10:20:56
## 25:                                     b 2018-06-15 10:20:56
## 26:                                 runif 2018-06-15 10:20:56
## 27:                                   688 2018-06-15 10:20:56
## 28:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 29:                   withCallingHandlers 2018-06-15 10:20:56
## 30:                               saveRDS 2018-06-15 10:20:56
## 31:                               do.call 2018-06-15 10:20:56
## 32:                          process_file 2018-06-15 10:20:56
## 33:                         process_group 2018-06-15 10:20:56
## 34:                   process_group.block 2018-06-15 10:20:56
## 35:                            call_block 2018-06-15 10:20:56
## 36:                            block_exec 2018-06-15 10:20:56
## 37:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:56
## 38: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-06-15 10:20:56
##                                  tagValue         createdDate
showCache(tmpDir, userTags = c("^a$")) # regular expression ... "a" exactly
## Cache size:
##   Total (including Rasters): 20.8 Kb
##   Selected objects (not including Rasters): 171 bytes
##                             artifact         tagKey
##  1: f3be0cb1ac973869cb34739d2d3fbedd         format
##  2: f3be0cb1ac973869cb34739d2d3fbedd           name
##  3: f3be0cb1ac973869cb34739d2d3fbedd          class
##  4: f3be0cb1ac973869cb34739d2d3fbedd           date
##  5: f3be0cb1ac973869cb34739d2d3fbedd        cacheId
##  6: f3be0cb1ac973869cb34739d2d3fbedd     objectName
##  7: f3be0cb1ac973869cb34739d2d3fbedd       function
##  8: f3be0cb1ac973869cb34739d2d3fbedd    object.size
##  9: f3be0cb1ac973869cb34739d2d3fbedd       accessed
## 10: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 11: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 12: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 13: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 14: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 15: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 16: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 17: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 18: f3be0cb1ac973869cb34739d2d3fbedd      preDigest
## 19: f3be0cb1ac973869cb34739d2d3fbedd      preDigest
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:56
##  2:                                 Cache 2018-06-15 10:20:56
##  3:                               numeric 2018-06-15 10:20:56
##  4:                   2018-06-15 10:20:56 2018-06-15 10:20:56
##  5:      85874f26b2e0c1ef689a7d379d275ebf 2018-06-15 10:20:56
##  6:                                     a 2018-06-15 10:20:56
##  7:                                 rnorm 2018-06-15 10:20:56
##  8:                                   688 2018-06-15 10:20:56
##  9:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 10:                   withCallingHandlers 2018-06-15 10:20:56
## 11:                               saveRDS 2018-06-15 10:20:56
## 12:                               do.call 2018-06-15 10:20:56
## 13:                          process_file 2018-06-15 10:20:56
## 14:                         process_group 2018-06-15 10:20:56
## 15:                   process_group.block 2018-06-15 10:20:56
## 16:                            call_block 2018-06-15 10:20:56
## 17:                            block_exec 2018-06-15 10:20:56
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:56
## 19: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:56
showCache(tmpDir, userTags = c("runif")) # show only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 20.8 Kb
##   Selected objects (not including Rasters): 167 bytes
##                             artifact         tagKey
##  1: f86f822df16a2ae18b84e7877ebfe51f         format
##  2: f86f822df16a2ae18b84e7877ebfe51f           name
##  3: f86f822df16a2ae18b84e7877ebfe51f          class
##  4: f86f822df16a2ae18b84e7877ebfe51f           date
##  5: f86f822df16a2ae18b84e7877ebfe51f        cacheId
##  6: f86f822df16a2ae18b84e7877ebfe51f     objectName
##  7: f86f822df16a2ae18b84e7877ebfe51f       function
##  8: f86f822df16a2ae18b84e7877ebfe51f    object.size
##  9: f86f822df16a2ae18b84e7877ebfe51f       accessed
## 10: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 11: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 12: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 13: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 14: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 15: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 16: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 17: f86f822df16a2ae18b84e7877ebfe51f otherFunctions
## 18: f86f822df16a2ae18b84e7877ebfe51f      preDigest
## 19: f86f822df16a2ae18b84e7877ebfe51f      preDigest
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:56
##  2:                                 Cache 2018-06-15 10:20:56
##  3:                               numeric 2018-06-15 10:20:56
##  4:                   2018-06-15 10:20:56 2018-06-15 10:20:56
##  5:      e37bb635c97bc2eeecab63816b881bbc 2018-06-15 10:20:56
##  6:                                     b 2018-06-15 10:20:56
##  7:                                 runif 2018-06-15 10:20:56
##  8:                                   688 2018-06-15 10:20:56
##  9:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 10:                   withCallingHandlers 2018-06-15 10:20:56
## 11:                               saveRDS 2018-06-15 10:20:56
## 12:                               do.call 2018-06-15 10:20:56
## 13:                          process_file 2018-06-15 10:20:56
## 14:                         process_group 2018-06-15 10:20:56
## 15:                   process_group.block 2018-06-15 10:20:56
## 16:                            call_block 2018-06-15 10:20:56
## 17:                            block_exec 2018-06-15 10:20:56
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:56
## 19: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-06-15 10:20:56
clearCache(tmpDir, userTags = c("runif")) # remove only cached objects made during runif call
## Cache size:
##   Total (including Rasters): 20.8 Kb
##   Selected objects (not including Rasters): 167 bytes
showCache(tmpDir) # only those made during rnorm call
## Cache size:
##   Total (including Rasters): 20.6 Kb
##   Selected objects (not including Rasters): 615 bytes
##                             artifact         tagKey
##  1: 4488f48ce2dec423f25fdb815c7954e0         format
##  2: 4488f48ce2dec423f25fdb815c7954e0           name
##  3: 4488f48ce2dec423f25fdb815c7954e0          class
##  4: 4488f48ce2dec423f25fdb815c7954e0           date
##  5: 4488f48ce2dec423f25fdb815c7954e0        cacheId
##  6: 4488f48ce2dec423f25fdb815c7954e0       function
##  7: 4488f48ce2dec423f25fdb815c7954e0    object.size
##  8: 4488f48ce2dec423f25fdb815c7954e0       accessed
##  9: 4488f48ce2dec423f25fdb815c7954e0 otherFunctions
## 10: 4488f48ce2dec423f25fdb815c7954e0 otherFunctions
## 11: 4488f48ce2dec423f25fdb815c7954e0 otherFunctions
## 12: 4488f48ce2dec423f25fdb815c7954e0 otherFunctions
## 13: 4488f48ce2dec423f25fdb815c7954e0 otherFunctions
## 14: 4488f48ce2dec423f25fdb815c7954e0 otherFunctions
## 15: 4488f48ce2dec423f25fdb815c7954e0 otherFunctions
## 16: 4488f48ce2dec423f25fdb815c7954e0 otherFunctions
## 17: 4488f48ce2dec423f25fdb815c7954e0      preDigest
## 18: 4488f48ce2dec423f25fdb815c7954e0      preDigest
## 19: 4488f48ce2dec423f25fdb815c7954e0      preDigest
## 20: 4488f48ce2dec423f25fdb815c7954e0       accessed
## 21: 4488f48ce2dec423f25fdb815c7954e0       accessed
## 22: 4488f48ce2dec423f25fdb815c7954e0       accessed
## 23: 4e58d7f630971fc2d6962f8fa2961b6c         format
## 24: 4e58d7f630971fc2d6962f8fa2961b6c           name
## 25: 4e58d7f630971fc2d6962f8fa2961b6c          class
## 26: 4e58d7f630971fc2d6962f8fa2961b6c           date
## 27: 4e58d7f630971fc2d6962f8fa2961b6c        cacheId
## 28: 4e58d7f630971fc2d6962f8fa2961b6c       function
## 29: 4e58d7f630971fc2d6962f8fa2961b6c    object.size
## 30: 4e58d7f630971fc2d6962f8fa2961b6c       accessed
## 31: 4e58d7f630971fc2d6962f8fa2961b6c otherFunctions
## 32: 4e58d7f630971fc2d6962f8fa2961b6c otherFunctions
## 33: 4e58d7f630971fc2d6962f8fa2961b6c otherFunctions
## 34: 4e58d7f630971fc2d6962f8fa2961b6c otherFunctions
## 35: 4e58d7f630971fc2d6962f8fa2961b6c otherFunctions
## 36: 4e58d7f630971fc2d6962f8fa2961b6c otherFunctions
## 37: 4e58d7f630971fc2d6962f8fa2961b6c otherFunctions
## 38: 4e58d7f630971fc2d6962f8fa2961b6c otherFunctions
## 39: 4e58d7f630971fc2d6962f8fa2961b6c      preDigest
## 40: 4e58d7f630971fc2d6962f8fa2961b6c      preDigest
## 41: 4e58d7f630971fc2d6962f8fa2961b6c      preDigest
## 42: f3be0cb1ac973869cb34739d2d3fbedd         format
## 43: f3be0cb1ac973869cb34739d2d3fbedd           name
## 44: f3be0cb1ac973869cb34739d2d3fbedd          class
## 45: f3be0cb1ac973869cb34739d2d3fbedd           date
## 46: f3be0cb1ac973869cb34739d2d3fbedd        cacheId
## 47: f3be0cb1ac973869cb34739d2d3fbedd     objectName
## 48: f3be0cb1ac973869cb34739d2d3fbedd       function
## 49: f3be0cb1ac973869cb34739d2d3fbedd    object.size
## 50: f3be0cb1ac973869cb34739d2d3fbedd       accessed
## 51: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 52: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 53: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 54: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 55: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 56: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 57: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 58: f3be0cb1ac973869cb34739d2d3fbedd otherFunctions
## 59: f3be0cb1ac973869cb34739d2d3fbedd      preDigest
## 60: f3be0cb1ac973869cb34739d2d3fbedd      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:55
##  2:                                 Cache 2018-06-15 10:20:55
##  3:                               numeric 2018-06-15 10:20:55
##  4:                   2018-06-15 10:20:55 2018-06-15 10:20:55
##  5:      f185e4f8cea576a595f7b81be17e63b5 2018-06-15 10:20:55
##  6:                                 rnorm 2018-06-15 10:20:55
##  7:                                   784 2018-06-15 10:20:55
##  8:                   2018-06-15 10:20:55 2018-06-15 10:20:55
##  9:                   withCallingHandlers 2018-06-15 10:20:55
## 10:                               saveRDS 2018-06-15 10:20:55
## 11:                               do.call 2018-06-15 10:20:55
## 12:                          process_file 2018-06-15 10:20:55
## 13:                         process_group 2018-06-15 10:20:55
## 14:                   process_group.block 2018-06-15 10:20:55
## 15:                            call_block 2018-06-15 10:20:56
## 16:                            block_exec 2018-06-15 10:20:56
## 17:    n:52ad08b1270ae9be9c0a12805d408433 2018-06-15 10:20:56
## 18: mean:2c67b8e97ab3ea8d032fea4318223887 2018-06-15 10:20:56
## 19: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:56
## 20:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 21:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 22:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 23:                                   rda 2018-06-15 10:20:56
## 24:                                 Cache 2018-06-15 10:20:56
## 25:                               numeric 2018-06-15 10:20:56
## 26:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 27:      778e2e6b9a967175d1f08811a995269d 2018-06-15 10:20:56
## 28:                   rnorm pipe sequence 2018-06-15 10:20:56
## 29:                                   784 2018-06-15 10:20:56
## 30:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 31:                   withCallingHandlers 2018-06-15 10:20:56
## 32:                               saveRDS 2018-06-15 10:20:56
## 33:                               do.call 2018-06-15 10:20:56
## 34:                          process_file 2018-06-15 10:20:56
## 35:                         process_group 2018-06-15 10:20:56
## 36:                   process_group.block 2018-06-15 10:20:56
## 37:                            call_block 2018-06-15 10:20:56
## 38:                            block_exec 2018-06-15 10:20:56
## 39:    n:52ad08b1270ae9be9c0a12805d408433 2018-06-15 10:20:56
## 40: mean:6f529546a7c8cf6edda6081007b3d58d 2018-06-15 10:20:56
## 41: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:56
## 42:                                   rda 2018-06-15 10:20:56
## 43:                                 Cache 2018-06-15 10:20:56
## 44:                               numeric 2018-06-15 10:20:56
## 45:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 46:      85874f26b2e0c1ef689a7d379d275ebf 2018-06-15 10:20:56
## 47:                                     a 2018-06-15 10:20:56
## 48:                                 rnorm 2018-06-15 10:20:56
## 49:                                   688 2018-06-15 10:20:56
## 50:                   2018-06-15 10:20:56 2018-06-15 10:20:56
## 51:                   withCallingHandlers 2018-06-15 10:20:56
## 52:                               saveRDS 2018-06-15 10:20:56
## 53:                               do.call 2018-06-15 10:20:56
## 54:                          process_file 2018-06-15 10:20:56
## 55:                         process_group 2018-06-15 10:20:56
## 56:                   process_group.block 2018-06-15 10:20:56
## 57:                            call_block 2018-06-15 10:20:56
## 58:                            block_exec 2018-06-15 10:20:56
## 59:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:56
## 60: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:56
##                                  tagValue         createdDate
clearCache(tmpDir)

Example 2: using the “accessed” tag

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# access it again, from Cache
ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
##   loading cached result from previous rnorm call, adding to memoised copy
wholeCache <- showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 337 bytes
# keep only items accessed "recently" (i.e., only objectName:a)
onlyRecentlyAccessed <- showCache(tmpDir, userTags = max(wholeCache[tagKey == "accessed"]$tagValue))
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 337 bytes
# inverse join with 2 data.tables ... using: a[!b]
# i.e., return all of wholeCache that was not recently accessed
toRemove <- unique(wholeCache[!onlyRecentlyAccessed], by = "artifact")$artifact
clearCache(tmpDir, toRemove) # remove ones not recently accessed
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 337 bytes
showCache(tmpDir) # still has more recently accessed
## Cache size:
##   Total (including Rasters): 12 Kb
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate
clearCache(tmpDir)

Example 3: using keepCache

ranNumsA <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# keep only those cached items from the last 24 hours
oneDay <- 60 * 60 * 24
keepCache(tmpDir, after = Sys.time() - oneDay)
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 335 bytes
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 335 bytes
##                             artifact         tagKey
##  1: 2e0856107759c9b5cdef091df3c11e83         format
##  2: 2e0856107759c9b5cdef091df3c11e83           name
##  3: 2e0856107759c9b5cdef091df3c11e83          class
##  4: 2e0856107759c9b5cdef091df3c11e83           date
##  5: 2e0856107759c9b5cdef091df3c11e83        cacheId
##  6: 2e0856107759c9b5cdef091df3c11e83     objectName
##  7: 2e0856107759c9b5cdef091df3c11e83       function
##  8: 2e0856107759c9b5cdef091df3c11e83    object.size
##  9: 2e0856107759c9b5cdef091df3c11e83       accessed
## 10: 2e0856107759c9b5cdef091df3c11e83 otherFunctions
## 11: 2e0856107759c9b5cdef091df3c11e83 otherFunctions
## 12: 2e0856107759c9b5cdef091df3c11e83 otherFunctions
## 13: 2e0856107759c9b5cdef091df3c11e83 otherFunctions
## 14: 2e0856107759c9b5cdef091df3c11e83 otherFunctions
## 15: 2e0856107759c9b5cdef091df3c11e83 otherFunctions
## 16: 2e0856107759c9b5cdef091df3c11e83 otherFunctions
## 17: 2e0856107759c9b5cdef091df3c11e83 otherFunctions
## 18: 2e0856107759c9b5cdef091df3c11e83      preDigest
## 19: 2e0856107759c9b5cdef091df3c11e83      preDigest
## 20: 5ecd34a4be4808fbc2e9891d33c1cf66         format
## 21: 5ecd34a4be4808fbc2e9891d33c1cf66           name
## 22: 5ecd34a4be4808fbc2e9891d33c1cf66          class
## 23: 5ecd34a4be4808fbc2e9891d33c1cf66           date
## 24: 5ecd34a4be4808fbc2e9891d33c1cf66        cacheId
## 25: 5ecd34a4be4808fbc2e9891d33c1cf66     objectName
## 26: 5ecd34a4be4808fbc2e9891d33c1cf66       function
## 27: 5ecd34a4be4808fbc2e9891d33c1cf66    object.size
## 28: 5ecd34a4be4808fbc2e9891d33c1cf66       accessed
## 29: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 30: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 31: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 32: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 33: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 34: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 35: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 36: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 37: 5ecd34a4be4808fbc2e9891d33c1cf66      preDigest
## 38: 5ecd34a4be4808fbc2e9891d33c1cf66      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:57
##  2:                                 Cache 2018-06-15 10:20:57
##  3:                               numeric 2018-06-15 10:20:57
##  4:                   2018-06-15 10:20:57 2018-06-15 10:20:57
##  5:      e37bb635c97bc2eeecab63816b881bbc 2018-06-15 10:20:57
##  6:                                     b 2018-06-15 10:20:57
##  7:                                 runif 2018-06-15 10:20:57
##  8:                                   688 2018-06-15 10:20:57
##  9:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 10:                   withCallingHandlers 2018-06-15 10:20:57
## 11:                               saveRDS 2018-06-15 10:20:57
## 12:                               do.call 2018-06-15 10:20:57
## 13:                          process_file 2018-06-15 10:20:57
## 14:                         process_group 2018-06-15 10:20:57
## 15:                   process_group.block 2018-06-15 10:20:57
## 16:                            call_block 2018-06-15 10:20:57
## 17:                            block_exec 2018-06-15 10:20:57
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:57
## 19: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-06-15 10:20:57
## 20:                                   rda 2018-06-15 10:20:57
## 21:                                 Cache 2018-06-15 10:20:57
## 22:                               numeric 2018-06-15 10:20:57
## 23:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 24:      85874f26b2e0c1ef689a7d379d275ebf 2018-06-15 10:20:57
## 25:                                     a 2018-06-15 10:20:57
## 26:                                 rnorm 2018-06-15 10:20:57
## 27:                                   688 2018-06-15 10:20:57
## 28:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 29:                   withCallingHandlers 2018-06-15 10:20:57
## 30:                               saveRDS 2018-06-15 10:20:57
## 31:                               do.call 2018-06-15 10:20:57
## 32:                          process_file 2018-06-15 10:20:57
## 33:                         process_group 2018-06-15 10:20:57
## 34:                   process_group.block 2018-06-15 10:20:57
## 35:                            call_block 2018-06-15 10:20:57
## 36:                            block_exec 2018-06-15 10:20:57
## 37:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:57
## 38: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:57
##                                  tagValue         createdDate
# Keep all Cache items created with an rnorm() call
keepCache(tmpDir, userTags = "rnorm")
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 335 bytes
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 171 bytes
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 164 bytes
##                             artifact         tagKey
##  1: 5ecd34a4be4808fbc2e9891d33c1cf66         format
##  2: 5ecd34a4be4808fbc2e9891d33c1cf66           name
##  3: 5ecd34a4be4808fbc2e9891d33c1cf66          class
##  4: 5ecd34a4be4808fbc2e9891d33c1cf66           date
##  5: 5ecd34a4be4808fbc2e9891d33c1cf66        cacheId
##  6: 5ecd34a4be4808fbc2e9891d33c1cf66     objectName
##  7: 5ecd34a4be4808fbc2e9891d33c1cf66       function
##  8: 5ecd34a4be4808fbc2e9891d33c1cf66    object.size
##  9: 5ecd34a4be4808fbc2e9891d33c1cf66       accessed
## 10: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 11: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 12: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 13: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 14: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 15: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 16: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 17: 5ecd34a4be4808fbc2e9891d33c1cf66 otherFunctions
## 18: 5ecd34a4be4808fbc2e9891d33c1cf66      preDigest
## 19: 5ecd34a4be4808fbc2e9891d33c1cf66      preDigest
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:57
##  2:                                 Cache 2018-06-15 10:20:57
##  3:                               numeric 2018-06-15 10:20:57
##  4:                   2018-06-15 10:20:57 2018-06-15 10:20:57
##  5:      85874f26b2e0c1ef689a7d379d275ebf 2018-06-15 10:20:57
##  6:                                     a 2018-06-15 10:20:57
##  7:                                 rnorm 2018-06-15 10:20:57
##  8:                                   688 2018-06-15 10:20:57
##  9:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 10:                   withCallingHandlers 2018-06-15 10:20:57
## 11:                               saveRDS 2018-06-15 10:20:57
## 12:                               do.call 2018-06-15 10:20:57
## 13:                          process_file 2018-06-15 10:20:57
## 14:                         process_group 2018-06-15 10:20:57
## 15:                   process_group.block 2018-06-15 10:20:57
## 16:                            call_block 2018-06-15 10:20:57
## 17:                            block_exec 2018-06-15 10:20:57
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:57
## 19: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:57
# Remove all Cache items that happened within a rnorm() call
clearCache(tmpDir, userTags = "rnorm")
## Cache size:
##   Total (including Rasters): 12.2 Kb
##   Selected objects (not including Rasters): 171 bytes
showCache(tmpDir) ## empty
## Cache size:
##   Total (including Rasters): 12 Kb
##   Selected objects (not including Rasters): 0 bytes
## Empty data.table (0 rows) of 3 cols: md5hash,name,createdDate

Example 4: searching for multiple objects in the cache

# default userTags is "and" matching; for "or" matching use |
ranNumsA <- Cache(runif, 4, cacheRepo = tmpDir, userTags = "objectName:a")
ranNumsB <- Cache(rnorm, 4, cacheRepo = tmpDir, userTags = "objectName:b")

# show all objects (runif and rnorm in this case)
showCache(tmpDir)
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 337 bytes
##                             artifact         tagKey
##  1: 9d08079bc8cc23273e0b4d4bef7b6511         format
##  2: 9d08079bc8cc23273e0b4d4bef7b6511           name
##  3: 9d08079bc8cc23273e0b4d4bef7b6511          class
##  4: 9d08079bc8cc23273e0b4d4bef7b6511           date
##  5: 9d08079bc8cc23273e0b4d4bef7b6511        cacheId
##  6: 9d08079bc8cc23273e0b4d4bef7b6511     objectName
##  7: 9d08079bc8cc23273e0b4d4bef7b6511       function
##  8: 9d08079bc8cc23273e0b4d4bef7b6511    object.size
##  9: 9d08079bc8cc23273e0b4d4bef7b6511       accessed
## 10: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 11: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 12: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 13: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 14: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 15: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 16: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 17: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 18: 9d08079bc8cc23273e0b4d4bef7b6511      preDigest
## 19: 9d08079bc8cc23273e0b4d4bef7b6511      preDigest
## 20: be0127621f56caff0f2be2c7eaff5cf7         format
## 21: be0127621f56caff0f2be2c7eaff5cf7           name
## 22: be0127621f56caff0f2be2c7eaff5cf7          class
## 23: be0127621f56caff0f2be2c7eaff5cf7           date
## 24: be0127621f56caff0f2be2c7eaff5cf7        cacheId
## 25: be0127621f56caff0f2be2c7eaff5cf7     objectName
## 26: be0127621f56caff0f2be2c7eaff5cf7       function
## 27: be0127621f56caff0f2be2c7eaff5cf7    object.size
## 28: be0127621f56caff0f2be2c7eaff5cf7       accessed
## 29: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 30: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 31: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 32: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 33: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 34: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 35: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 36: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 37: be0127621f56caff0f2be2c7eaff5cf7      preDigest
## 38: be0127621f56caff0f2be2c7eaff5cf7      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:57
##  2:                                 Cache 2018-06-15 10:20:57
##  3:                               numeric 2018-06-15 10:20:57
##  4:                   2018-06-15 10:20:57 2018-06-15 10:20:57
##  5:      e37bb635c97bc2eeecab63816b881bbc 2018-06-15 10:20:57
##  6:                                     a 2018-06-15 10:20:57
##  7:                                 runif 2018-06-15 10:20:57
##  8:                                   688 2018-06-15 10:20:57
##  9:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 10:                   withCallingHandlers 2018-06-15 10:20:57
## 11:                               saveRDS 2018-06-15 10:20:57
## 12:                               do.call 2018-06-15 10:20:57
## 13:                          process_file 2018-06-15 10:20:57
## 14:                         process_group 2018-06-15 10:20:57
## 15:                   process_group.block 2018-06-15 10:20:57
## 16:                            call_block 2018-06-15 10:20:57
## 17:                            block_exec 2018-06-15 10:20:57
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:57
## 19: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-06-15 10:20:57
## 20:                                   rda 2018-06-15 10:20:57
## 21:                                 Cache 2018-06-15 10:20:57
## 22:                               numeric 2018-06-15 10:20:57
## 23:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 24:      85874f26b2e0c1ef689a7d379d275ebf 2018-06-15 10:20:57
## 25:                                     b 2018-06-15 10:20:57
## 26:                                 rnorm 2018-06-15 10:20:57
## 27:                                   688 2018-06-15 10:20:57
## 28:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 29:                   withCallingHandlers 2018-06-15 10:20:57
## 30:                               saveRDS 2018-06-15 10:20:57
## 31:                               do.call 2018-06-15 10:20:57
## 32:                          process_file 2018-06-15 10:20:57
## 33:                         process_group 2018-06-15 10:20:57
## 34:                   process_group.block 2018-06-15 10:20:57
## 35:                            call_block 2018-06-15 10:20:57
## 36:                            block_exec 2018-06-15 10:20:57
## 37:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:57
## 38: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:57
##                                  tagValue         createdDate
# show objects that are both runif and rnorm
# (i.e., none in this case, because objecs are either or, not both)
showCache(tmpDir, userTags = c("runif", "rnorm")) ## empty
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 337 bytes
## Empty data.table (0 rows) of 4 cols: artifact,tagKey,tagValue,createdDate
# show objects that are either runif or rnorm ("or" search)
showCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 337 bytes
##                             artifact         tagKey
##  1: 9d08079bc8cc23273e0b4d4bef7b6511         format
##  2: 9d08079bc8cc23273e0b4d4bef7b6511           name
##  3: 9d08079bc8cc23273e0b4d4bef7b6511          class
##  4: 9d08079bc8cc23273e0b4d4bef7b6511           date
##  5: 9d08079bc8cc23273e0b4d4bef7b6511        cacheId
##  6: 9d08079bc8cc23273e0b4d4bef7b6511     objectName
##  7: 9d08079bc8cc23273e0b4d4bef7b6511       function
##  8: 9d08079bc8cc23273e0b4d4bef7b6511    object.size
##  9: 9d08079bc8cc23273e0b4d4bef7b6511       accessed
## 10: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 11: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 12: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 13: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 14: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 15: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 16: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 17: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 18: 9d08079bc8cc23273e0b4d4bef7b6511      preDigest
## 19: 9d08079bc8cc23273e0b4d4bef7b6511      preDigest
## 20: be0127621f56caff0f2be2c7eaff5cf7         format
## 21: be0127621f56caff0f2be2c7eaff5cf7           name
## 22: be0127621f56caff0f2be2c7eaff5cf7          class
## 23: be0127621f56caff0f2be2c7eaff5cf7           date
## 24: be0127621f56caff0f2be2c7eaff5cf7        cacheId
## 25: be0127621f56caff0f2be2c7eaff5cf7     objectName
## 26: be0127621f56caff0f2be2c7eaff5cf7       function
## 27: be0127621f56caff0f2be2c7eaff5cf7    object.size
## 28: be0127621f56caff0f2be2c7eaff5cf7       accessed
## 29: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 30: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 31: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 32: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 33: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 34: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 35: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 36: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 37: be0127621f56caff0f2be2c7eaff5cf7      preDigest
## 38: be0127621f56caff0f2be2c7eaff5cf7      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:57
##  2:                                 Cache 2018-06-15 10:20:57
##  3:                               numeric 2018-06-15 10:20:57
##  4:                   2018-06-15 10:20:57 2018-06-15 10:20:57
##  5:      e37bb635c97bc2eeecab63816b881bbc 2018-06-15 10:20:57
##  6:                                     a 2018-06-15 10:20:57
##  7:                                 runif 2018-06-15 10:20:57
##  8:                                   688 2018-06-15 10:20:57
##  9:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 10:                   withCallingHandlers 2018-06-15 10:20:57
## 11:                               saveRDS 2018-06-15 10:20:57
## 12:                               do.call 2018-06-15 10:20:57
## 13:                          process_file 2018-06-15 10:20:57
## 14:                         process_group 2018-06-15 10:20:57
## 15:                   process_group.block 2018-06-15 10:20:57
## 16:                            call_block 2018-06-15 10:20:57
## 17:                            block_exec 2018-06-15 10:20:57
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:57
## 19: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-06-15 10:20:57
## 20:                                   rda 2018-06-15 10:20:57
## 21:                                 Cache 2018-06-15 10:20:57
## 22:                               numeric 2018-06-15 10:20:57
## 23:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 24:      85874f26b2e0c1ef689a7d379d275ebf 2018-06-15 10:20:57
## 25:                                     b 2018-06-15 10:20:57
## 26:                                 rnorm 2018-06-15 10:20:57
## 27:                                   688 2018-06-15 10:20:57
## 28:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 29:                   withCallingHandlers 2018-06-15 10:20:57
## 30:                               saveRDS 2018-06-15 10:20:57
## 31:                               do.call 2018-06-15 10:20:57
## 32:                          process_file 2018-06-15 10:20:57
## 33:                         process_group 2018-06-15 10:20:57
## 34:                   process_group.block 2018-06-15 10:20:57
## 35:                            call_block 2018-06-15 10:20:57
## 36:                            block_exec 2018-06-15 10:20:57
## 37:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:57
## 38: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:57
##                                  tagValue         createdDate
# keep only objects that are either runif or rnorm ("or" search)
keepCache(tmpDir, userTags = "runif|rnorm")
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 337 bytes
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 337 bytes
##                             artifact         tagKey
##  1: 9d08079bc8cc23273e0b4d4bef7b6511         format
##  2: 9d08079bc8cc23273e0b4d4bef7b6511           name
##  3: 9d08079bc8cc23273e0b4d4bef7b6511          class
##  4: 9d08079bc8cc23273e0b4d4bef7b6511           date
##  5: 9d08079bc8cc23273e0b4d4bef7b6511        cacheId
##  6: 9d08079bc8cc23273e0b4d4bef7b6511     objectName
##  7: 9d08079bc8cc23273e0b4d4bef7b6511       function
##  8: 9d08079bc8cc23273e0b4d4bef7b6511    object.size
##  9: 9d08079bc8cc23273e0b4d4bef7b6511       accessed
## 10: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 11: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 12: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 13: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 14: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 15: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 16: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 17: 9d08079bc8cc23273e0b4d4bef7b6511 otherFunctions
## 18: 9d08079bc8cc23273e0b4d4bef7b6511      preDigest
## 19: 9d08079bc8cc23273e0b4d4bef7b6511      preDigest
## 20: be0127621f56caff0f2be2c7eaff5cf7         format
## 21: be0127621f56caff0f2be2c7eaff5cf7           name
## 22: be0127621f56caff0f2be2c7eaff5cf7          class
## 23: be0127621f56caff0f2be2c7eaff5cf7           date
## 24: be0127621f56caff0f2be2c7eaff5cf7        cacheId
## 25: be0127621f56caff0f2be2c7eaff5cf7     objectName
## 26: be0127621f56caff0f2be2c7eaff5cf7       function
## 27: be0127621f56caff0f2be2c7eaff5cf7    object.size
## 28: be0127621f56caff0f2be2c7eaff5cf7       accessed
## 29: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 30: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 31: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 32: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 33: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 34: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 35: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 36: be0127621f56caff0f2be2c7eaff5cf7 otherFunctions
## 37: be0127621f56caff0f2be2c7eaff5cf7      preDigest
## 38: be0127621f56caff0f2be2c7eaff5cf7      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:57
##  2:                                 Cache 2018-06-15 10:20:57
##  3:                               numeric 2018-06-15 10:20:57
##  4:                   2018-06-15 10:20:57 2018-06-15 10:20:57
##  5:      e37bb635c97bc2eeecab63816b881bbc 2018-06-15 10:20:57
##  6:                                     a 2018-06-15 10:20:57
##  7:                                 runif 2018-06-15 10:20:57
##  8:                                   688 2018-06-15 10:20:57
##  9:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 10:                   withCallingHandlers 2018-06-15 10:20:57
## 11:                               saveRDS 2018-06-15 10:20:57
## 12:                               do.call 2018-06-15 10:20:57
## 13:                          process_file 2018-06-15 10:20:57
## 14:                         process_group 2018-06-15 10:20:57
## 15:                   process_group.block 2018-06-15 10:20:57
## 16:                            call_block 2018-06-15 10:20:57
## 17:                            block_exec 2018-06-15 10:20:57
## 18:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:57
## 19: .FUN:d2631d24c3b38b89c7bdd4ab7faaaac3 2018-06-15 10:20:57
## 20:                                   rda 2018-06-15 10:20:57
## 21:                                 Cache 2018-06-15 10:20:57
## 22:                               numeric 2018-06-15 10:20:57
## 23:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 24:      85874f26b2e0c1ef689a7d379d275ebf 2018-06-15 10:20:57
## 25:                                     b 2018-06-15 10:20:57
## 26:                                 rnorm 2018-06-15 10:20:57
## 27:                                   688 2018-06-15 10:20:57
## 28:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 29:                   withCallingHandlers 2018-06-15 10:20:57
## 30:                               saveRDS 2018-06-15 10:20:57
## 31:                               do.call 2018-06-15 10:20:57
## 32:                          process_file 2018-06-15 10:20:57
## 33:                         process_group 2018-06-15 10:20:57
## 34:                   process_group.block 2018-06-15 10:20:57
## 35:                            call_block 2018-06-15 10:20:57
## 36:                            block_exec 2018-06-15 10:20:57
## 37:    n:969a49ec15bcd4323ff31538af321264 2018-06-15 10:20:57
## 38: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:57
##                                  tagValue         createdDate
clearCache(tmpDir)

Example 5: using caching to speed up rerunning expensive computations

ras <- raster(extent(0, 5, 0, 5), res = 1,
              vals = sample(1:5, replace = TRUE, size = 25),
              crs = "+proj=lcc +lat_1=48 +lat_2=33 +lon_0=-100 +ellps=WGS84")

# A slow operation, like GIS operation
notCached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  projectRaster(ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

cached <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  # using quote works also
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)

# second time is much faster
reRun <- suppressWarnings(
  # project raster generates warnings when run non-interactively
  Cache(projectRaster, ras, crs = crs(ras), res = 5, cacheRepo = tmpDir)
)
##   loading cached result from previous projectRaster call, adding to memoised copy
# recovered cached version is same as non-cached version
all.equal(notCached, reRun) ## TRUE
## [1] TRUE

Nested Caching

Nested caching, which is when Caching of a function occurs inside an outer function, which is itself cached. This is a critical element to working within a reproducible work flow. It is not enough during development to cache flat code chunks, as there will be many levels of “slow” functions. Ideally, at all points in a development cycle, it should be possible to get to any line of code starting from the very initial steps, running through everything up to that point, in less that 1 second. If the workflow can be kept very fast like this, then there is a guarantee that it will work at any point.

##########################
## Nested Caching
# Make 2 functions
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean)
}
outer <- function(n) {
  Cache(inner, 0.1, cacheRepo = tmpdir2)
}

# make 2 different cache paths
tmpdir1 <- file.path(tempdir(), "first")
tmpdir2 <- file.path(tempdir(), "second")

# Run the Cache ... notOlderThan propagates to all 3 Cache calls,
#   but cacheRepo is tmpdir1 in top level Cache and all nested
#   Cache calls, unless individually overridden ... here inner
#   uses tmpdir2 repository
Cache(outer, n = 2, cacheRepo = tmpdir1, notOlderThan = Sys.time())
## [1]  0.2718205 -1.2807258 -1.5240252
## attr(,"tags")
## [1] "cacheId:e09bf93970f9e94d5c639cfa8ca722f0"
## attr(,"newCache")
## [1] TRUE
## attr(,"call")
## [1] ""
showCache(tmpdir1) # 2 function calls
## Cache size:
##   Total (including Rasters): 12.3 Kb
##   Selected objects (not including Rasters): 327 bytes
##                             artifact         tagKey
##  1: 38e32468e4816d595244ebe58ce1c6a1         format
##  2: 38e32468e4816d595244ebe58ce1c6a1           name
##  3: 38e32468e4816d595244ebe58ce1c6a1          class
##  4: 38e32468e4816d595244ebe58ce1c6a1           date
##  5: 38e32468e4816d595244ebe58ce1c6a1        cacheId
##  6: 38e32468e4816d595244ebe58ce1c6a1       function
##  7: 38e32468e4816d595244ebe58ce1c6a1    object.size
##  8: 38e32468e4816d595244ebe58ce1c6a1       accessed
##  9: 38e32468e4816d595244ebe58ce1c6a1 otherFunctions
## 10: 38e32468e4816d595244ebe58ce1c6a1 otherFunctions
## 11: 38e32468e4816d595244ebe58ce1c6a1 otherFunctions
## 12: 38e32468e4816d595244ebe58ce1c6a1 otherFunctions
## 13: 38e32468e4816d595244ebe58ce1c6a1 otherFunctions
## 14: 38e32468e4816d595244ebe58ce1c6a1 otherFunctions
## 15: 38e32468e4816d595244ebe58ce1c6a1 otherFunctions
## 16: 38e32468e4816d595244ebe58ce1c6a1 otherFunctions
## 17: 38e32468e4816d595244ebe58ce1c6a1      preDigest
## 18: 38e32468e4816d595244ebe58ce1c6a1      preDigest
## 19: e97d3e11d86080c45103290e9e016d02         format
## 20: e97d3e11d86080c45103290e9e016d02           name
## 21: e97d3e11d86080c45103290e9e016d02          class
## 22: e97d3e11d86080c45103290e9e016d02           date
## 23: e97d3e11d86080c45103290e9e016d02        cacheId
## 24: e97d3e11d86080c45103290e9e016d02       function
## 25: e97d3e11d86080c45103290e9e016d02    object.size
## 26: e97d3e11d86080c45103290e9e016d02       accessed
## 27: e97d3e11d86080c45103290e9e016d02 otherFunctions
## 28: e97d3e11d86080c45103290e9e016d02 otherFunctions
## 29: e97d3e11d86080c45103290e9e016d02 otherFunctions
## 30: e97d3e11d86080c45103290e9e016d02 otherFunctions
## 31: e97d3e11d86080c45103290e9e016d02 otherFunctions
## 32: e97d3e11d86080c45103290e9e016d02 otherFunctions
## 33: e97d3e11d86080c45103290e9e016d02 otherFunctions
## 34: e97d3e11d86080c45103290e9e016d02 otherFunctions
## 35: e97d3e11d86080c45103290e9e016d02      preDigest
## 36: e97d3e11d86080c45103290e9e016d02      preDigest
## 37: e97d3e11d86080c45103290e9e016d02      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:58
##  2:                                 Cache 2018-06-15 10:20:58
##  3:                               numeric 2018-06-15 10:20:58
##  4:                   2018-06-15 10:20:58 2018-06-15 10:20:58
##  5:      e09bf93970f9e94d5c639cfa8ca722f0 2018-06-15 10:20:58
##  6:                                 outer 2018-06-15 10:20:58
##  7:                                   688 2018-06-15 10:20:58
##  8:                   2018-06-15 10:20:58 2018-06-15 10:20:58
##  9:                   withCallingHandlers 2018-06-15 10:20:58
## 10:                               saveRDS 2018-06-15 10:20:58
## 11:                               do.call 2018-06-15 10:20:58
## 12:                          process_file 2018-06-15 10:20:58
## 13:                         process_group 2018-06-15 10:20:58
## 14:                   process_group.block 2018-06-15 10:20:58
## 15:                            call_block 2018-06-15 10:20:58
## 16:                            block_exec 2018-06-15 10:20:58
## 17:    n:8128a6180a705341ab7c05cfa945edfb 2018-06-15 10:20:58
## 18: .FUN:b5f6bcbdd9f23e39c2c5d4020e73a6ff 2018-06-15 10:20:58
## 19:                                   rda 2018-06-15 10:20:57
## 20:                                 Cache 2018-06-15 10:20:57
## 21:                               numeric 2018-06-15 10:20:57
## 22:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 23:      cec73d63ad3864af8bcd7efc5fae864d 2018-06-15 10:20:57
## 24:                                 rnorm 2018-06-15 10:20:57
## 25:                                   688 2018-06-15 10:20:57
## 26:                   2018-06-15 10:20:57 2018-06-15 10:20:57
## 27:                   withCallingHandlers 2018-06-15 10:20:57
## 28:                               saveRDS 2018-06-15 10:20:57
## 29:                               do.call 2018-06-15 10:20:57
## 30:                          process_file 2018-06-15 10:20:57
## 31:                         process_group 2018-06-15 10:20:57
## 32:                   process_group.block 2018-06-15 10:20:57
## 33:                            call_block 2018-06-15 10:20:57
## 34:                            block_exec 2018-06-15 10:20:57
## 35:    n:4ae3e6b6364de42fdc243469d73448cc 2018-06-15 10:20:57
## 36: mean:c28b87a0be6a99966bdaa5e556974b43 2018-06-15 10:20:57
## 37: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:57
##                                  tagValue         createdDate
showCache(tmpdir2) # 1 function call
## Cache size:
##   Total (including Rasters): 12.2 Kb
##   Selected objects (not including Rasters): 164 bytes
##                             artifact         tagKey
##  1: 386864e0001f2eff2d817f6881bb3d9f         format
##  2: 386864e0001f2eff2d817f6881bb3d9f           name
##  3: 386864e0001f2eff2d817f6881bb3d9f          class
##  4: 386864e0001f2eff2d817f6881bb3d9f           date
##  5: 386864e0001f2eff2d817f6881bb3d9f        cacheId
##  6: 386864e0001f2eff2d817f6881bb3d9f       function
##  7: 386864e0001f2eff2d817f6881bb3d9f    object.size
##  8: 386864e0001f2eff2d817f6881bb3d9f       accessed
##  9: 386864e0001f2eff2d817f6881bb3d9f otherFunctions
## 10: 386864e0001f2eff2d817f6881bb3d9f otherFunctions
## 11: 386864e0001f2eff2d817f6881bb3d9f otherFunctions
## 12: 386864e0001f2eff2d817f6881bb3d9f otherFunctions
## 13: 386864e0001f2eff2d817f6881bb3d9f otherFunctions
## 14: 386864e0001f2eff2d817f6881bb3d9f otherFunctions
## 15: 386864e0001f2eff2d817f6881bb3d9f otherFunctions
## 16: 386864e0001f2eff2d817f6881bb3d9f otherFunctions
## 17: 386864e0001f2eff2d817f6881bb3d9f      preDigest
## 18: 386864e0001f2eff2d817f6881bb3d9f      preDigest
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:57
##  2:                                 Cache 2018-06-15 10:20:57
##  3:                               numeric 2018-06-15 10:20:57
##  4:                   2018-06-15 10:20:57 2018-06-15 10:20:57
##  5:      19b808ac6871e0184e63c421a116cb61 2018-06-15 10:20:57
##  6:                                 inner 2018-06-15 10:20:57
##  7:                                   688 2018-06-15 10:20:57
##  8:                   2018-06-15 10:20:57 2018-06-15 10:20:57
##  9:                   withCallingHandlers 2018-06-15 10:20:57
## 10:                               saveRDS 2018-06-15 10:20:58
## 11:                               do.call 2018-06-15 10:20:58
## 12:                          process_file 2018-06-15 10:20:58
## 13:                         process_group 2018-06-15 10:20:58
## 14:                   process_group.block 2018-06-15 10:20:58
## 15:                            call_block 2018-06-15 10:20:58
## 16:                            block_exec 2018-06-15 10:20:58
## 17: mean:c28b87a0be6a99966bdaa5e556974b43 2018-06-15 10:20:58
## 18: .FUN:56a1302d7ef43383766d7af6ca072c4e 2018-06-15 10:20:58
# userTags get appended
# all items have the outer tag propagate, plus inner ones only have inner ones
clearCache(tmpdir1)
outerTag <- "outerTag"
innerTag <- "innerTag"
inner <- function(mean) {
  d <- 1
  Cache(rnorm, n = 3, mean = mean, notOlderThan = Sys.time() - 1e5, userTags = innerTag)
}
outer <- function(n) {
  Cache(inner, 0.1)
}
aa <- Cache(outer, n = 2, cacheRepo = tmpdir1, userTags = outerTag)
showCache(tmpdir1) # rnorm function has outerTag and innerTag, inner and outer only have outerTag
## Cache size:
##   Total (including Rasters): 20.5 Kb
##   Selected objects (not including Rasters): 494 bytes
##                             artifact         tagKey
##  1: 0847b45b97ef65b942209adf094277d7         format
##  2: 0847b45b97ef65b942209adf094277d7           name
##  3: 0847b45b97ef65b942209adf094277d7          class
##  4: 0847b45b97ef65b942209adf094277d7           date
##  5: 0847b45b97ef65b942209adf094277d7        cacheId
##  6: 0847b45b97ef65b942209adf094277d7       outerTag
##  7: 0847b45b97ef65b942209adf094277d7       function
##  8: 0847b45b97ef65b942209adf094277d7    object.size
##  9: 0847b45b97ef65b942209adf094277d7       accessed
## 10: 0847b45b97ef65b942209adf094277d7 otherFunctions
## 11: 0847b45b97ef65b942209adf094277d7 otherFunctions
## 12: 0847b45b97ef65b942209adf094277d7 otherFunctions
## 13: 0847b45b97ef65b942209adf094277d7 otherFunctions
## 14: 0847b45b97ef65b942209adf094277d7 otherFunctions
## 15: 0847b45b97ef65b942209adf094277d7 otherFunctions
## 16: 0847b45b97ef65b942209adf094277d7 otherFunctions
## 17: 0847b45b97ef65b942209adf094277d7 otherFunctions
## 18: 0847b45b97ef65b942209adf094277d7      preDigest
## 19: 0847b45b97ef65b942209adf094277d7      preDigest
## 20: e24471853eb9555fdb51940ab76d70a5         format
## 21: e24471853eb9555fdb51940ab76d70a5           name
## 22: e24471853eb9555fdb51940ab76d70a5          class
## 23: e24471853eb9555fdb51940ab76d70a5           date
## 24: e24471853eb9555fdb51940ab76d70a5        cacheId
## 25: e24471853eb9555fdb51940ab76d70a5       outerTag
## 26: e24471853eb9555fdb51940ab76d70a5       function
## 27: e24471853eb9555fdb51940ab76d70a5    object.size
## 28: e24471853eb9555fdb51940ab76d70a5       accessed
## 29: e24471853eb9555fdb51940ab76d70a5 otherFunctions
## 30: e24471853eb9555fdb51940ab76d70a5 otherFunctions
## 31: e24471853eb9555fdb51940ab76d70a5 otherFunctions
## 32: e24471853eb9555fdb51940ab76d70a5 otherFunctions
## 33: e24471853eb9555fdb51940ab76d70a5 otherFunctions
## 34: e24471853eb9555fdb51940ab76d70a5 otherFunctions
## 35: e24471853eb9555fdb51940ab76d70a5 otherFunctions
## 36: e24471853eb9555fdb51940ab76d70a5 otherFunctions
## 37: e24471853eb9555fdb51940ab76d70a5      preDigest
## 38: e24471853eb9555fdb51940ab76d70a5      preDigest
## 39: f8999e8ae0c3a7c57eb3e079adaf97f9         format
## 40: f8999e8ae0c3a7c57eb3e079adaf97f9           name
## 41: f8999e8ae0c3a7c57eb3e079adaf97f9          class
## 42: f8999e8ae0c3a7c57eb3e079adaf97f9           date
## 43: f8999e8ae0c3a7c57eb3e079adaf97f9        cacheId
## 44: f8999e8ae0c3a7c57eb3e079adaf97f9       innerTag
## 45: f8999e8ae0c3a7c57eb3e079adaf97f9       outerTag
## 46: f8999e8ae0c3a7c57eb3e079adaf97f9       function
## 47: f8999e8ae0c3a7c57eb3e079adaf97f9    object.size
## 48: f8999e8ae0c3a7c57eb3e079adaf97f9       accessed
## 49: f8999e8ae0c3a7c57eb3e079adaf97f9 otherFunctions
## 50: f8999e8ae0c3a7c57eb3e079adaf97f9 otherFunctions
## 51: f8999e8ae0c3a7c57eb3e079adaf97f9 otherFunctions
## 52: f8999e8ae0c3a7c57eb3e079adaf97f9 otherFunctions
## 53: f8999e8ae0c3a7c57eb3e079adaf97f9 otherFunctions
## 54: f8999e8ae0c3a7c57eb3e079adaf97f9 otherFunctions
## 55: f8999e8ae0c3a7c57eb3e079adaf97f9 otherFunctions
## 56: f8999e8ae0c3a7c57eb3e079adaf97f9 otherFunctions
## 57: f8999e8ae0c3a7c57eb3e079adaf97f9      preDigest
## 58: f8999e8ae0c3a7c57eb3e079adaf97f9      preDigest
## 59: f8999e8ae0c3a7c57eb3e079adaf97f9      preDigest
##                             artifact         tagKey
##                                  tagValue         createdDate
##  1:                                   rda 2018-06-15 10:20:58
##  2:                                 Cache 2018-06-15 10:20:58
##  3:                               numeric 2018-06-15 10:20:58
##  4:                   2018-06-15 10:20:58 2018-06-15 10:20:58
##  5:      994d1330fbd961f795ab0dc508271963 2018-06-15 10:20:58
##  6:                              outerTag 2018-06-15 10:20:58
##  7:                                 inner 2018-06-15 10:20:58
##  8:                                   688 2018-06-15 10:20:58
##  9:                   2018-06-15 10:20:58 2018-06-15 10:20:58
## 10:                   withCallingHandlers 2018-06-15 10:20:58
## 11:                               saveRDS 2018-06-15 10:20:58
## 12:                               do.call 2018-06-15 10:20:58
## 13:                          process_file 2018-06-15 10:20:58
## 14:                         process_group 2018-06-15 10:20:58
## 15:                   process_group.block 2018-06-15 10:20:58
## 16:                            call_block 2018-06-15 10:20:58
## 17:                            block_exec 2018-06-15 10:20:58
## 18: mean:c28b87a0be6a99966bdaa5e556974b43 2018-06-15 10:20:58
## 19: .FUN:b910401646b09073940de757678db03d 2018-06-15 10:20:58
## 20:                                   rda 2018-06-15 10:20:58
## 21:                                 Cache 2018-06-15 10:20:58
## 22:                               numeric 2018-06-15 10:20:58
## 23:                   2018-06-15 10:20:58 2018-06-15 10:20:58
## 24:      44f57deb36c53cd9c395e04c51fea77a 2018-06-15 10:20:58
## 25:                              outerTag 2018-06-15 10:20:58
## 26:                                 outer 2018-06-15 10:20:58
## 27:                                   688 2018-06-15 10:20:58
## 28:                   2018-06-15 10:20:58 2018-06-15 10:20:58
## 29:                   withCallingHandlers 2018-06-15 10:20:58
## 30:                               saveRDS 2018-06-15 10:20:58
## 31:                               do.call 2018-06-15 10:20:58
## 32:                          process_file 2018-06-15 10:20:58
## 33:                         process_group 2018-06-15 10:20:58
## 34:                   process_group.block 2018-06-15 10:20:58
## 35:                            call_block 2018-06-15 10:20:58
## 36:                            block_exec 2018-06-15 10:20:58
## 37:    n:8128a6180a705341ab7c05cfa945edfb 2018-06-15 10:20:58
## 38: .FUN:62302feda89e19149a56ca40fde725e1 2018-06-15 10:20:58
## 39:                                   rda 2018-06-15 10:20:58
## 40:                                 Cache 2018-06-15 10:20:58
## 41:                               numeric 2018-06-15 10:20:58
## 42:                   2018-06-15 10:20:58 2018-06-15 10:20:58
## 43:      cec73d63ad3864af8bcd7efc5fae864d 2018-06-15 10:20:58
## 44:                              innerTag 2018-06-15 10:20:58
## 45:                              outerTag 2018-06-15 10:20:58
## 46:                                 rnorm 2018-06-15 10:20:58
## 47:                                   688 2018-06-15 10:20:58
## 48:                   2018-06-15 10:20:58 2018-06-15 10:20:58
## 49:                   withCallingHandlers 2018-06-15 10:20:58
## 50:                               saveRDS 2018-06-15 10:20:58
## 51:                               do.call 2018-06-15 10:20:58
## 52:                          process_file 2018-06-15 10:20:58
## 53:                         process_group 2018-06-15 10:20:58
## 54:                   process_group.block 2018-06-15 10:20:58
## 55:                            call_block 2018-06-15 10:20:58
## 56:                            block_exec 2018-06-15 10:20:58
## 57:    n:4ae3e6b6364de42fdc243469d73448cc 2018-06-15 10:20:58
## 58: mean:c28b87a0be6a99966bdaa5e556974b43 2018-06-15 10:20:58
## 59: .FUN:7e9a928f110f80b3612e71883a6ec1f4 2018-06-15 10:20:58
##                                  tagValue         createdDate

cacheId

Sometimes, it is not absolutely desirable to maintain the work flow intact because changes that are irrelevant to the analysis, such as changing messages sent to a user, may be changed, without a desire to rerun functions. The cacheId argument is for this. Once a piece of code is run, then the cacheId can be manually extracted (it is reported at the end of a Cache call) and manually placed in the code, passed in as, say, cacheId = "ad184ce64541972b50afd8e7b75f821b".

### cacheId
set.seed(1)
Cache(rnorm, 1, cacheRepo = tmpdir1)
## [1] -0.6264538
## attr(,"tags")
## [1] "cacheId:23dc247384c1b270f0d36de4bca1b138"
## attr(,"newCache")
## [1] TRUE
## attr(,"call")
## [1] ""
# manually look at output attribute which shows cacheId: ad184ce64541972b50afd8e7b75f821b
Cache(rnorm, 1, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b") # same value
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
## [1] 0.1836433
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"newCache")
## [1] TRUE
## attr(,"call")
## [1] ""
# override even with different inputs:
Cache(rnorm, 2, cacheRepo = tmpdir1, cacheId = "ad184ce64541972b50afd8e7b75f821b")
## cacheId is not same as calculated hash. Manually searching for cacheId:ad184ce64541972b50afd8e7b75f821b
##   loading cached result from previous rnorm call, adding to memoised copy
## [1] 0.1836433
## attr(,"tags")
## [1] "cacheId:ad184ce64541972b50afd8e7b75f821b"
## attr(,"newCache")
## [1] FALSE
## attr(,"call")
## [1] ""
## cleanup
unlink(c("filename.rda", "filename1.rda"))

Working with the Cache manually

Since the cache is simply an archivist repository, all archivist functions will work as is. In addition, there are several helpers in the reproducible package, including showCache, keepCache and clearCache that may be useful. Also, one can access cached items manually (rather than simply rerunning the same Cache function again).

if (requireNamespace("archivist")) {
  # get the RasterLayer that was produced with the gaussMap function:
  mapHash <- unique(showCache(tmpDir, userTags = "projectRaster")$artifact)
  map <- archivist::loadFromLocalRepo(md5hash = mapHash[1], repoDir = tmpDir, value = TRUE)
  
  plot(map)
}
## Cache size:
##   Total (including Rasters): 12.8 Kb
##   Selected objects (not including Rasters): 807 bytes

## cleanup
unlink(dirname(tmpDir), recursive = TRUE)

Reproducible Workflow

In general, we feel that a liberal use of Cache will make a re-usable and reproducible work flow. shiny apps can be made, taking advantage of Cache. Indeed, much of the difficulty in managing data sets and saving them for future use, can be accommodated by caching.

Nested Caching

Cache individual functions

Cache(<functionName>, <other arguments>)

This will allow fine scale control of individual function calls.