DOI

Overview

tidyfst is a toolkit of tidy data manipulation verbs with data.table as the backend . Combining the merits of syntax elegance from dplyr and computing performance from data.table, tidyfst intends to provide users with state-of-the-art data manipulation tools with least pain. This package is inspired by maditr, but follows a different philosophy of design, such as prohibiting in place replacement and used a "_dt" suffix API. Also, tidyfst would introduce more tidy data verbs from other packages, including but not limited to tidyverse and data.table. If you are a dplyr user but have to use data.table for speedy computation, or data.table user looking for readable coding syntax, tidyfst is designed for you (and me of course). For further details and tutorials, see vignettes.

Enjoy the data science in tidyfst !

Features

  • Receives any data.frame (tibble/data.table/data.frame) and returns a data.table.
  • Show the variable class of data.table as default.
  • Never use in place replacement.
  • Use suffix rather than prefix to increase the efficiency (especially when you have IDE with automatic code completion).
  • More flexible verbs for big data manipulation.
  • Supporting data importing and parsing with fst, details see parse_fst/select_fst/filter_fst and import_fst/export_fst.
  • Flagship functions: group_dt, unnest_dt, mutate_when, etc.

Installation

install.packages("tidyfst")
# or
devtools::install_github("hope-data-science/tidyfst")

Future plans

unnest_dt is now fast enough to beat the tidyr::unnest, but the nest_dt function would build a nested data.table with data.table inside. How to use such data structure is remained to be seen, and the performance is still to be explored.

Acknowledgement

The author of maditr, Gregory Demin and the author of fst, Marcus Klik have helped me a lot in the development of this work. It is so lucky to have them (and many other selfless contributors) in the same open source community of R.