Published April 14, 2022
| Version 2.1.0
Software
Open
huggingface/datasets: 2.1.0
Authors/Creators
- Quentin Lhoest1
-
Albert Villanova del Moral1
- Patrick von Platen1
- Thomas Wolf1
- Mario Šaško1
- Yacine Jernite1
- Abhishek Thakur1
- Lewis Tunstall1
- Suraj Patil1
- Mariama Drame1
- Julien Chaumond1
- Julien Plu1
- Joe Davison1
- Simon Brandeis1
- Victor Sanh1
- Teven Le Scao1
- Kevin Canwen Xu1
- Nicolas Patry1
- Steven Liu1
- Angelina McMillan-Major1
- Philipp Schmid1
- Sylvain Gugger1
- Nathan Raw1
- Sylvain Lesage1
- Anton Lozhkov1
- Matthew Carrigan1
- Théo Matussière1
- Leandro von Werra1
- Lysandre Debut1
- Stas Bekman1
- Clément Delangue1
- 1. Hugging Face
Description
Datasets Changes
- New:initial monash time series forecasting repository by @kashif in https://github.com/huggingface/datasets/pull/3743
- Update: Wikipedia by @albertvillanova in https://github.com/huggingface/datasets/pull/3821 and https://github.com/huggingface/datasets/pull/3989
- Update: conll2012_ontonotesv5 - Support streaming by @albertvillanova in https://github.com/huggingface/datasets/pull/4002
- Update: daily_dialog - Support streaming by @albertvillanova in https://github.com/huggingface/datasets/pull/4008
- Update: id_clickbait - Support streaming by @albertvillanova in https://github.com/huggingface/datasets/pull/4014
- Update: blimp - Support streaming by @albertvillanova in https://github.com/huggingface/datasets/pull/4016
- Update: scan - Support streaming by @albertvillanova in https://github.com/huggingface/datasets/pull/4017
- Update: yelp_review_full - Replace data url by @lhoestq in https://github.com/huggingface/datasets/pull/4018
- Update: yelp_polarity - Support streaming by @lhoestq in https://github.com/huggingface/datasets/pull/4019
- Update: amazon_polarity - Replace data URL by @lhoestq in https://github.com/huggingface/datasets/pull/4020
- Update: dbpedia_14 - Replace data url by @lhoestq in https://github.com/huggingface/datasets/pull/4022
- New: Roman Urdu Hate Speech dataset by @bp-high in https://github.com/huggingface/datasets/pull/3972
- Update: xtreme - Support streaming dataset for bucc18 config by @albertvillanova in https://github.com/huggingface/datasets/pull/4026
- Fix: cats_vs_dogs - fix checksum error dataset by @albertvillanova in https://github.com/huggingface/datasets/pull/4033
- Fix: xcopa - fix null checksum by @albertvillanova in https://github.com/huggingface/datasets/pull/4034
- Update: yahoo_answers_topics - Replace data url by @lhoestq in https://github.com/huggingface/datasets/pull/4023
- New: Adversarial GLUE by @jxmorris12 in https://github.com/huggingface/datasets/pull/3849
- Update: ASSIN 2 dataset - replace broken Google Drive URLS by links on github by @ruanchaves in https://github.com/huggingface/datasets/pull/4004
- Update: xcopa - Support streaming by @albertvillanova in https://github.com/huggingface/datasets/pull/4039
- Deprecated: Multilingual Librispeech - deprecate dataset in favor of
facebook/multilingual_librispeechby @polinaeterna in https://github.com/huggingface/datasets/pull/4060 - New: MetaShift by @dnaveenr in https://github.com/huggingface/datasets/pull/3900
- Fix: amazon_us_reviews - fix metadata - 4/4/2022 by @trentonstrong in https://github.com/huggingface/datasets/pull/4092
- Update: medical_dialog - Add configs with processed data by @albertvillanova in https://github.com/huggingface/datasets/pull/4127
- Update: xtreme - Support streaming for udpos config by @albertvillanova in https://github.com/huggingface/datasets/pull/4131
- Update: xtreme - Support streaming for PAWS-X config by @albertvillanova in https://github.com/huggingface/datasets/pull/4132
- Update: xtreme - Support streaming for PAN-X config by @albertvillanova in https://github.com/huggingface/datasets/pull/4135
- New: GSM8K by @jon-tow in https://github.com/huggingface/datasets/pull/4103
- New: SBU Captions Photo by @thomasw21 in https://github.com/huggingface/datasets/pull/4130
- Update: SQuAD v2 - Use a constant for the articles regex by @bryant1410 in https://github.com/huggingface/datasets/pull/4030
- Update: HANS - Support streaming by @mariosasko in https://github.com/huggingface/datasets/pull/4155
- Update (BREAKING): TIMIT - Redirect users to download data manually from LDC by @lhoestq in https://github.com/huggingface/datasets/pull/4145
- Updated annotations for nli_tr dataset by @e-budur in https://github.com/huggingface/datasets/pull/4058
- Add missing label for emotion description by @lijiazheng99 in https://github.com/huggingface/datasets/pull/4151
- Remove unncessary 'pylint disable' message in ReadMe by @Datta0 in https://github.com/huggingface/datasets/pull/3955
- Improve RedCaps dataset card by @mariosasko in https://github.com/huggingface/datasets/pull/4100
- Fix duplicate key in multi_news by @lhoestq in https://github.com/huggingface/datasets/pull/4164
- Tasks alignment with models by @lhoestq in https://github.com/huggingface/datasets/pull/4066
- Update datasets task tags to align tags with models by @lhoestq in https://github.com/huggingface/datasets/pull/4067
- Xtreme-S Metrics by @patrickvonplaten in https://github.com/huggingface/datasets/pull/3799
- Fix xtreme s metrics by @patrickvonplaten in https://github.com/huggingface/datasets/pull/3957
- Avoid info log messages from transformers in FrugalScore metric by @albertvillanova in https://github.com/huggingface/datasets/pull/3938
- Add exact match metric by @emibaylor in https://github.com/huggingface/datasets/pull/3899
- Fix comet metric by @lhoestq in https://github.com/huggingface/datasets/pull/3945
- Add zero_division argument to precision and recall metrics by @albertvillanova in https://github.com/huggingface/datasets/pull/4035
- Support float data types in pearsonr/spearmanr metrics by @albertvillanova in https://github.com/huggingface/datasets/pull/4054
- Remove GLEU metric by @emibaylor in https://github.com/huggingface/datasets/pull/3949
- Perplexity Metric Card by @emibaylor in https://github.com/huggingface/datasets/pull/3905
- Create README.md by @sashavor in https://github.com/huggingface/datasets/pull/3917
- Create README.md for CER metric by @sashavor in https://github.com/huggingface/datasets/pull/3911
- Create README.md by @sashavor in https://github.com/huggingface/datasets/pull/3944
- Update README.md by @sashavor in https://github.com/huggingface/datasets/pull/3933
- Create SARI metric card by @sashavor in https://github.com/huggingface/datasets/pull/3932
- Create MAUVE metric card by @sashavor in https://github.com/huggingface/datasets/pull/3934
- Create CoVAL metric card by @sashavor in https://github.com/huggingface/datasets/pull/3940
- Google BLEU Metric Card by @emibaylor in https://github.com/huggingface/datasets/pull/3948
- Create metric card for BERTScore by @sashavor in https://github.com/huggingface/datasets/pull/3966
- Rename wer to cer by @pmgautam in https://github.com/huggingface/datasets/pull/4012
- Create metric card for XNLI by @sashavor in https://github.com/huggingface/datasets/pull/4046
- Create metric card for the Code Eval metric by @sashavor in https://github.com/huggingface/datasets/pull/4049
- Add TER metric card by @emibaylor in https://github.com/huggingface/datasets/pull/3981
- BLEU metric card by @emibaylor in https://github.com/huggingface/datasets/pull/3947
- Create metric card for CUAD by @sashavor in https://github.com/huggingface/datasets/pull/4043
- Create metric card for METEOR by @sashavor in https://github.com/huggingface/datasets/pull/4065
- Create a metric card for Competition MATH by @sashavor in https://github.com/huggingface/datasets/pull/4073
- Create metric card for seqeval by @sashavor in https://github.com/huggingface/datasets/pull/4070
- Create README.md by @sashavor in https://github.com/huggingface/datasets/pull/3930
- Create metric card for Frugal Score by @sashavor in https://github.com/huggingface/datasets/pull/4089
- Updating FrugalScore metric card by @sashavor in https://github.com/huggingface/datasets/pull/4097
- Proposing WikiSplit metric card by @sashavor in https://github.com/huggingface/datasets/pull/4098
- Fix formatting in BLEU metric card by @mariosasko in https://github.com/huggingface/datasets/pull/4157
- Doc maintenance by @stevhliu in https://github.com/huggingface/datasets/pull/3926
- [Doc] Don't use v for version tags on GitHub by @sgugger in https://github.com/huggingface/datasets/pull/3943
- Use templates for doc-builidng jobs by @sgugger in https://github.com/huggingface/datasets/pull/3914
- Add align_labels_with_mapping docs by @stevhliu in https://github.com/huggingface/datasets/pull/3931
- Add tip on how to speed up loading with ImageFolder by @mariosasko in https://github.com/huggingface/datasets/pull/3980
- Fix main_classes docs index by @lhoestq in https://github.com/huggingface/datasets/pull/3925
- More consistent references in docs by @mariosasko in https://github.com/huggingface/datasets/pull/3988
- Docs maintenance by @stevhliu in https://github.com/huggingface/datasets/pull/3999
- Add ROUGE Metric Card by @emibaylor in https://github.com/huggingface/datasets/pull/4076
- Add chrF(++) Metric Card by @emibaylor in https://github.com/huggingface/datasets/pull/4082
- Add SacreBLEU Metric Card by @emibaylor in https://github.com/huggingface/datasets/pull/4083
- Fix flatten of complex feature types by @mariosasko in https://github.com/huggingface/datasets/pull/3723
- Fix flatten of Sequence feature type by @lhoestq in https://github.com/huggingface/datasets/pull/3962
- Exclude Google Drive tests of the CI by @lhoestq in https://github.com/huggingface/datasets/pull/3982
- Close
PIL.Imagefile handler inImage.decode_exampleby @mariosasko in https://github.com/huggingface/datasets/pull/3995 - Fix Faiss custom_index device by @albertvillanova in https://github.com/huggingface/datasets/pull/3987
- Fix None issue with Sequence of dict by @lhoestq in https://github.com/huggingface/datasets/pull/4010
- Update main readme by @lhoestq in https://github.com/huggingface/datasets/pull/3927
- Fix
mapremove_columns on empty dataset by @lhoestq in https://github.com/huggingface/datasets/pull/4021 - Fix Audio.encode_example() when writing an array by @polinaeterna in https://github.com/huggingface/datasets/pull/3998
- Use audio feature in ASR task template by @lhoestq in https://github.com/huggingface/datasets/pull/4006
- Improve out of bounds error message by @lhoestq in https://github.com/huggingface/datasets/pull/4068
- Increase max retries for GitHub metrics by @albertvillanova in https://github.com/huggingface/datasets/pull/4063
- Fix CLI dummy data generation by @albertvillanova in https://github.com/huggingface/datasets/pull/4045
- Fix docs on audio feature installation by @albertvillanova in https://github.com/huggingface/datasets/pull/4028
- Add installation instructions to image_process doc by @mariosasko in https://github.com/huggingface/datasets/pull/4072
- Fix GithubMetricModuleFactory instantiation with None download_config by @albertvillanova in https://github.com/huggingface/datasets/pull/4078
- Increase max retries for GitHub datasets by @albertvillanova in https://github.com/huggingface/datasets/pull/4079
- Close parquet writer properly in
push_to_hubby @lhoestq in https://github.com/huggingface/datasets/pull/4081 - fix typo in rename_column error message by @hunterlang in https://github.com/huggingface/datasets/pull/4095
- Fix BeamWriter output Parquet file by @albertvillanova in https://github.com/huggingface/datasets/pull/4087
- Remove unused legacy Beam utils by @albertvillanova in https://github.com/huggingface/datasets/pull/4088
- Hotfix failing CI tests on Windows by @albertvillanova in https://github.com/huggingface/datasets/pull/4119
- Update security policy by @albertvillanova in https://github.com/huggingface/datasets/pull/4111
- Avoid writing empty license files by @albertvillanova in https://github.com/huggingface/datasets/pull/4090
- Support huggingface_hub 0.5 by @lhoestq in https://github.com/huggingface/datasets/pull/4106
- Pretty print dataset info files by @mariosasko in https://github.com/huggingface/datasets/pull/4116
- Add single dataset citations for TweetEval by @gchhablani in https://github.com/huggingface/datasets/pull/4137
- Adjust path to datasets tutorial in How-To by @NimaBoscarino in https://github.com/huggingface/datasets/pull/4147
- Applied index-filters on scores in search.py. by @vishalsrao in https://github.com/huggingface/datasets/pull/3971
- More robust
cast_to_python_objectsinTypedSequenceby @mariosasko in https://github.com/huggingface/datasets/pull/4128 - Sync Features dictionaries by @mariosasko in https://github.com/huggingface/datasets/pull/3997
- Avoid rate limit in update hub repositories by @lhoestq in https://github.com/huggingface/datasets/pull/4167
- @bp-high made their first contribution in https://github.com/huggingface/datasets/pull/3972
- @ruanchaves made their first contribution in https://github.com/huggingface/datasets/pull/4004
- @pmgautam made their first contribution in https://github.com/huggingface/datasets/pull/4012
- @hunterlang made their first contribution in https://github.com/huggingface/datasets/pull/4095
- @trentonstrong made their first contribution in https://github.com/huggingface/datasets/pull/4092
- @NimaBoscarino made their first contribution in https://github.com/huggingface/datasets/pull/4147
- @jon-tow made their first contribution in https://github.com/huggingface/datasets/pull/4103
- @lijiazheng99 made their first contribution in https://github.com/huggingface/datasets/pull/4151
- @Datta0 made their first contribution in https://github.com/huggingface/datasets/pull/3955
- @vishalsrao made their first contribution in https://github.com/huggingface/datasets/pull/3971
Full Changelog: https://github.com/huggingface/datasets/compare/2.0.0...2.1.0
Files
huggingface/datasets-2.1.0.zip
Files
(50.3 MB)
| Name | Size | Download all |
|---|---|---|
|
md5:f8664570817846eae14193bf1da40dd7
|
50.3 MB | Preview Download |
Additional details
Related works
- Is supplement to
- https://github.com/huggingface/datasets/tree/2.1.0 (URL)