piggyback is basically a poor soul’s Git LFS. GitHub rejects commits containing files larger than 50 Mb. Git LFS is not only expensive, it also breaks GitHub’s collaborative model. (Someone wants to submit a PR with a simple edit to your docs, they cannot fork) Unlike Git LFS, piggyback doesn’t take over your standard git client, it just perches comfortably on the shoulders of your existing GitHub API. Data can be versioned by piggyback, but relative to git LFS versioning is less strict: uploads can be set as a new version or allowed to overwrite previously uploaded data. piggyback works with both public and private repositories.
Installation
You can install the development version from GitHub with:
Quickstart
See the piggyback vignette for details on authentication and additional package functionality.
Piggyback can download data attached to a release on any repository:
library(piggyback)
pb_download("data/mtcars.tsv.gz", repo = "cboettig/piggyback-tests", dest = tempdir())Downloading from private repos or uploading to any repo requires authentication, so be sure to set a GITHUB_TOKEN (or GITHUB_PAT) environmental variable, or include the .token argument. Omit the file name to download all attached objects. Omit the repository name to default to the current repository. See vignette or function documentation for details.
We can also upload data to any existing release (defaults to latest):
## We'll need some example data first.
## Pro tip: compress your tabular data to save space & speed upload/downloads
readr::write_tsv(mtcars, "mtcars.tsv.gz")
pb_upload("mtcars.tsv.gz", repo = "cboettig/piggyback-tests")Tracking data files
For a Git LFS style workflow, just specify the type of files you wish to track using pb_track(). Piggyback will retain a record of these files in a hidden .pbattributes file in your repository, and add these to .gitignore so you don’t accidentally commit them to GitHub. pb_trak will aslo return a list of such files that you can easily pass to pb_upload():
# track csv files, compressed data, and geotiff files:
pb_track(c("*.csv", "*.gz", "*.tif")) %>%
pb_upload()You can easily download the latest version of all data attached to a given release with pb_download() with no file argument (analgous to a git pull for data):
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.
