Published July 21, 2021
| Version 0.6.0
Software
Open
ropensci/targets: Improved Target Markdown, configurability, shortcutting, workspace/error flexibility
Authors/Creators
- 1. Eli Lilly and Company @EliLillyCo
- 2. University of Toronto
- 3. Manaaki Whenua
- 4. University of Cambridge
- 5. @cynkra
- 6. University of Leeds
- 7. @jumpingrivers
Description
targets 0.6.0
Bug fixes
- Allow
tar_poll()to lose and then regain connection to the progress file. - Make sure changes to the
tar_groupcolumn ofiteration = "group"data frames do not invalidate slices (#507, @lindsayplatt).
- In Target Markdown, add a new
tar_interactiveglobal option to select interactive mode or non-interactive mode (#469). - Highlight a graph neighborhood when the user clicks a node. Control the neighborhood degree with new arguments
degree_fromanddegree_tooftar_visnetwork()andtar_glimpse()(#474, @rgayler). - Make the target script path configurable in
tar_config_set()(#476). - Add a
tar_scriptchunk option in Target Markdown to control where the{targets}language engine writes the target script and helper scripts (#478). - Add new arguments
scriptandstoreto choose custom paths to the target script file and data store for individual function calls (#477). - Allow users to set an alternative path to the YAML configuration file for the current R session (#477). Most users have no reason to set this path, it is only for niche applications like Shiny apps with
targetsbackends. Unavoidably, the path gets reset to_targets.yamlwhen the session restarts. - Add new
_targets.yamlconfig optionsreporter_make,reporter_outdated, andworkersto control function argument defaults shared across multiple functions called outside_targets.R(#498, @ianeveperry). - Add
tar_load_globals()for debugging, testing, prototyping, and teaching (#496, @malcolmbarrett). - Add structure to the
resourcesargument oftar_target()to avoid conflicts among formats and HPC backends (#489). Includes user-side helper functions liketar_resources()andtar_resources_aws()to build the required data structures. - Log skipped targets in
_targets/meta/progressand display then intar_progress(),tar_poll(),tar_watch(),tar_progress_branches(),tar_progress_summary(), andtar_visnetwork()(#514). Instead of writing each skip line separately to_targets/meta/progress, accumulate skip lines in a queue and then write them all out in bulk when something interesting happens. This avoids a lot of overhead in certain cases. - Add a
shortcutargument totar_make(),tar_make_clustermq(),tar_make_future(),tar_outdated(), andtar_sitrep()to more efficiently skip parts of the pipeline (#522, #523, @jennysjaarda, @MilesMcBain, @kendonB). - Support
namesandshortcutin graph data frames and graph visuals (#529). - Move
allowandexcludeto the network behind the graph visuals rather than the visuals themselves (#529). - Add a new "progress" display to the
tar_watch()app to show verbose progress info and metadata. - Add a new
workspace_on_errorargument oftar_option_set()to supersedeerror = "workspace". Helps control workspace behavior independently of theerrorargument oftar_target()(#405, #533, #534, @mattwarkentin, @xinstein). - Implement
error = "abridge"intar_target()and related functions. If a target errors out with this option, the target itself stops, any currently running targets keeps, and no new targets launch after that (#533, #534, @xinstein). - Add a menu prompt to
tar_destroy()which can be suppressed withTAR_ASK = "false"(#542, @gofford). - Support functions
tar_older()andtar_newer()to help users identify and invalidate targets at regular times or intervals.
- In Target Markdown, deprecate the
targetschunk option in favor oftar_globals(#469). - Deprecate
error = "workspace"intar_target()and related functions. Usetar_option_set(workspace_on_error = TRUE)instead (#405, #533, @mattwarkentin, @xinstein).
- Reset the backoff upper bound when concluding a target or shutting down a
clustermqworker (@rich-payne). - Set more aggressive default backoff bound of 0.1 seconds (previous: 5 seconds) and set a more aggressive minimum of 0.001 seconds (previous: 0.01 seconds) (@rich-payne).
- Speed up the summary and forecast reporters by only printing to the console every quarter second.
- Avoid superfluous calls to
store_sync_file_meta.default()on small files. - In
tar_watch(), take several measures to avoid long computation times rendering the graph:- Expose arguments
displayanddisplaystotar_watch()so the user can select which display shows first. - Make
"summary"the default display instead of"graph". - Set
outdatedtoFALSEby default.
- Expose arguments
- Simplify the Target Markdown example.
- Warn about unnamed chunks in Target Markdown.
- Redesign option system to be more object-oriented and rigorous. Also export most options to HPC workers (#475).
- Simplify config system to let API function arguments take control (#483).
- In
tar_read()for targets withformat = "aws_file", download the file back to the path the user originally saved it when the target ran. - Replace the
TAR_MAKE_REPORTERenvironment variable withtargets::tar_config_get("reporter_make"). - Use
eval(parse(text = readLines("_targets.R")), envir = some_envir)and related techniques instead of the less controllablesource(). Expose anenvirargument to many functions for further control over evaluation ifcallr_functionisNULL. - Drop
out.attrswhen hashing groups of data frames to extend #507 toexpand.grid()(#508). - Increase the number of characters in errors and warnings up to 2048.
- Refactor assertions to automatically generate better messages.
- Export assertions, conditions, and language utilities in packages that build on top of
targets. - Change
GITHUBPATtoGITHUB_TOKENin thetar_github_actions()YAML file (#554, @eveyp). - Support the
evalchunk option in Target Markdown (#552, @fkohrt). - Record time stamps in the metadata
timecolumn for all builder targets, regardless of storage format.
Files
ropensci/targets-0.6.0.zip
Additional details
Related works
- Is supplement to
- https://github.com/ropensci/targets/tree/0.6.0 (URL)