There is a newer version of this record available.

Software Open Access

Transformers: State-of-the-Art Natural Language Processing

Wolf, Thomas; Debut, Lysandre; Sanh, Victor; Chaumond, Julien; Delangue, Clement; Moi, Anthony; Cistac, Perric; Ma, Clara; Jernite, Yacine; Plu, Julien; Xu, Canwen; Le Scao, Teven; Gugger, Sylvain; Drame, Mariama; Lhoest, Quentin; Rush, Alexander M.


Dublin Core Export

<?xml version='1.0' encoding='utf-8'?>
<oai_dc:dc xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
  <dc:creator>Wolf, Thomas</dc:creator>
  <dc:creator>Debut, Lysandre</dc:creator>
  <dc:creator>Sanh, Victor</dc:creator>
  <dc:creator>Chaumond, Julien</dc:creator>
  <dc:creator>Delangue, Clement</dc:creator>
  <dc:creator>Moi, Anthony</dc:creator>
  <dc:creator>Cistac, Perric</dc:creator>
  <dc:creator>Ma, Clara</dc:creator>
  <dc:creator>Jernite, Yacine</dc:creator>
  <dc:creator>Plu, Julien</dc:creator>
  <dc:creator>Xu, Canwen</dc:creator>
  <dc:creator>Le Scao, Teven</dc:creator>
  <dc:creator>Gugger, Sylvain</dc:creator>
  <dc:creator>Drame, Mariama</dc:creator>
  <dc:creator>Lhoest, Quentin</dc:creator>
  <dc:creator>Rush, Alexander M.</dc:creator>
  <dc:date>2020-10-01</dc:date>
  <dc:description>TrOCR and VisionEncoderDecoderModel
One new model is released as part of the TrOCR implementation: TrOCRForCausalLM, in PyTorch. It comes along a new VisionEncoderDecoderModel class, which allows to mix-and-match any vision Transformer encoder with any text Transformer as decoder, similar to the existing SpeechEncoderDecoderModel class.
The TrOCR model was proposed in TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models, by Minghao Li, Tengchao Lv, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, Furu Wei.
The TrOCR model consists of an image transformer encoder and an autoregressive text transformer to perform optical character recognition in an end-to-end manner.

Add TrOCR + VisionEncoderDecoderModel by @NielsRogge in https://github.com/huggingface/transformers/pull/13874

Compatible checkpoints can be found on the Hub: https://huggingface.co/models?other=trocr
SEW &amp; SEW-D
SEW and SEW-D (Squeezed and Efficient Wav2Vec) were proposed in Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition by Felix Wu, Kwangyoun Kim, Jing Pan, Kyu Han, Kilian Q. Weinberger, Yoav Artzi.
SEW and SEW-D models use a Wav2Vec-style feature encoder and introduce temporal downsampling to reduce the length of the transformer encoder. SEW-D additionally replaces the transformer encoder with a DeBERTa one. Both models achieve significant inference speedups without sacrificing the speech recognition quality.

Add the SEW and SEW-D speech models by @anton-l in https://github.com/huggingface/transformers/pull/13962
Add SEW CTC models by @anton-l in https://github.com/huggingface/transformers/pull/14158

Compatible checkpoints are available on the Hub: https://huggingface.co/models?other=sew and https://huggingface.co/models?other=sew-d
DistilHuBERT
DistilHuBERT was proposed in DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT, by Heng-Jui Chang, Shu-wen Yang, Hung-yi Lee.
DistilHuBERT is a distilled version of the HuBERT model. Using only two transformer layers, the model scores competitively on the SUPERB benchmark tasks.
Compatible checkpoint is available on the Hub: https://huggingface.co/ntu-spml/distilhubert
TensorFlow improvements
Several bug fixes and UX improvements for TensorFlow
Keras callback
Introduction of a Keras callback to push to the hub each epoch, or after a given number of steps:

Keras callback to push to hub each epoch, or after N steps by @Rocketknight1 in https://github.com/huggingface/transformers/pull/13773

Updates on the encoder-decoder framework
The encoder-decoder framework is now available in TensorFlow, allowing mixing and matching different encoders and decoders together into a single encoder-decoder architecture!

Add TFEncoderDecoderModel + Add cross-attention to some TF models by @ydshieh in https://github.com/huggingface/transformers/pull/13222

Besides this, the EncoderDecoderModel classes have been updated to work similar to models like BART and T5. From now on, users don't need to pass decoder_input_ids themselves anymore to the model. Instead, they will be created automatically based on the labels (namely by shifting them one position to the right, replacing -100 by the pad_token_id and prepending the decoder_start_token_id). Note that this may result in training discrepancies if fine-tuning a model trained with versions anterior to 4.12.0 that set the decoder_input_ids = labels.

Fix EncoderDecoderModel classes to be more like BART and T5 by @NielsRogge  in https://github.com/huggingface/transformers/pull/14139

Speech improvements

Add DistilHuBERT  by @anton-l in https://github.com/huggingface/transformers/pull/14174
[Speech Examples] Add pytorch speech pretraining by @patrickvonplaten in https://github.com/huggingface/transformers/pull/13877
[Speech Examples] Add new audio feature by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14027
Add ASR colabs by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14067
[ASR] Make speech recognition example more general to load any tokenizer by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14079
[Examples] Add an official audio classification example by @anton-l in https://github.com/huggingface/transformers/pull/13722
[Examples] Use Audio feature in speech classification by @anton-l in https://github.com/huggingface/transformers/pull/14052

Auto-model API
To make it easier to extend the Transformers library, every Auto class a new register method, that allows you to register your own custom models, configurations or tokenizers. See more in the documentation

Add an API to register objects to Auto classes by @sgugger in https://github.com/huggingface/transformers/pull/13989

Bug fixes and improvements

Fix filtering in test fetcher utils by @sgugger in https://github.com/huggingface/transformers/pull/13766
Fix warning for gradient_checkpointing by @sgugger in https://github.com/huggingface/transformers/pull/13767
Implement len in IterableDatasetShard by @sgugger in https://github.com/huggingface/transformers/pull/13780
[Wav2Vec2] Better error message by @patrickvonplaten in https://github.com/huggingface/transformers/pull/13777
Fix LayoutLM ONNX test error by @nishprabhu in https://github.com/huggingface/transformers/pull/13710
Enable readme link synchronization by @qqaatw in https://github.com/huggingface/transformers/pull/13785
Fix length of IterableDatasetShard and add test by @sgugger in https://github.com/huggingface/transformers/pull/13792
[docs/gpt-j] addd instructions for how minimize CPU RAM usage by @patil-suraj in https://github.com/huggingface/transformers/pull/13795
[examples run_glue.py] missing requirements scipy, sklearn by @stas00 in https://github.com/huggingface/transformers/pull/13768
[examples/flax] use Repository API for push_to_hub by @patil-suraj in https://github.com/huggingface/transformers/pull/13672
Fix gather for TPU by @sgugger in https://github.com/huggingface/transformers/pull/13813
[testing] auto-replay captured streams by @stas00 in https://github.com/huggingface/transformers/pull/13803
Add MultiBERTs conversion script by @gchhablani in https://github.com/huggingface/transformers/pull/13077
[Examples] Improve mapping in accelerate examples by @patrickvonplaten in https://github.com/huggingface/transformers/pull/13810
[DPR] Correct init by @patrickvonplaten in https://github.com/huggingface/transformers/pull/13796
skip gptj slow generate tests by @patil-suraj in https://github.com/huggingface/transformers/pull/13809
Fix warning situation: UserWarning: max_length is ignored when padding=True" by @shirayu in https://github.com/huggingface/transformers/pull/13829
Updating CITATION.cff to fix GitHub citation prompt BibTeX output. by @arfon in https://github.com/huggingface/transformers/pull/13833
Add TF notebooks by @Rocketknight1 in https://github.com/huggingface/transformers/pull/13793
Bart: check if decoder_inputs_embeds is set by @silviu-oprea in https://github.com/huggingface/transformers/pull/13800
include megatron_gpt2 in installed modules by @stas00 in https://github.com/huggingface/transformers/pull/13834
Delete MultiBERTs conversion script by @gchhablani in https://github.com/huggingface/transformers/pull/13852
Remove a duplicated bullet point in the GPT-J doc by @yaserabdelaziz in https://github.com/huggingface/transformers/pull/13851
Add Mistral GPT-2 Stability Tweaks by @siddk in https://github.com/huggingface/transformers/pull/13573
Fix broken link to distill models in docs by @Randl in https://github.com/huggingface/transformers/pull/13848
:sparkles: update image classification example by @nateraw in https://github.com/huggingface/transformers/pull/13824
Update no_* argument (HfArgumentParser) by @BramVanroy in https://github.com/huggingface/transformers/pull/13865
Update Tatoeba conversion by @Traubert in https://github.com/huggingface/transformers/pull/13757
Fixing 1-length special tokens cut. by @Narsil in https://github.com/huggingface/transformers/pull/13862
Fix flax summarization example: save checkpoint after each epoch and push checkpoint to the hub by @ydshieh in https://github.com/huggingface/transformers/pull/13872
Fixing empty prompts for text-generation when BOS exists. by @Narsil in https://github.com/huggingface/transformers/pull/13859
Improve error message when loading models from Hub by @aphedges in https://github.com/huggingface/transformers/pull/13836
Initial support for symbolic tracing with torch.fx allowing dynamic axes by @michaelbenayoun in https://github.com/huggingface/transformers/pull/13579
Allow dataset to be an optional argument for (Distributed)LengthGroupedSampler by @ZhaofengWu in https://github.com/huggingface/transformers/pull/13820
Fixing question-answering with long contexts  by @Narsil in https://github.com/huggingface/transformers/pull/13873
fix(integrations): consider test metrics by @borisdayma in https://github.com/huggingface/transformers/pull/13888
fix: replace asserts by value error by @m5l14i11 in https://github.com/huggingface/transformers/pull/13894
Update parallelism.md by @hyunwoongko in https://github.com/huggingface/transformers/pull/13892
Autodocument the list of ONNX-supported models by @sgugger in https://github.com/huggingface/transformers/pull/13884
Fixing GPU for token-classification in a better way. by @Narsil in https://github.com/huggingface/transformers/pull/13856
Update FSNER code in examples-&gt;research_projects-&gt;fsner by @sayef in https://github.com/huggingface/transformers/pull/13864
Replace assert statements with exceptions by @ddrm86 in https://github.com/huggingface/transformers/pull/13871
Fixing Backward compatiblity for zero-shot by @Narsil in https://github.com/huggingface/transformers/pull/13855
Update run_qa.py - CorrectTypo by @akulagrawal in https://github.com/huggingface/transformers/pull/13857
T5ForConditionalGeneration: enabling using past_key_values and labels in training by @yssjtu in https://github.com/huggingface/transformers/pull/13805
Fix trainer logging_nan_inf_filter in torch_xla mode by @ymwangg in https://github.com/huggingface/transformers/pull/13896
Fix hp search for non sigopt backends by @sgugger in https://github.com/huggingface/transformers/pull/13897
[Trainer] Fix nan-loss condition by @anton-l in https://github.com/huggingface/transformers/pull/13911
Raise exceptions instead of asserts in utils/download_glue_data by @hirotasoshu in https://github.com/huggingface/transformers/pull/13907
Add an example of exporting BartModel + BeamSearch to ONNX module. by @fatcat-z in https://github.com/huggingface/transformers/pull/13765
#12789 Replace assert statements with exceptions by @djroxx2000 in https://github.com/huggingface/transformers/pull/13909
Add missing whitespace to multiline strings by @aphedges in https://github.com/huggingface/transformers/pull/13916
[Wav2Vec2] Fix mask_feature_prob by @patrickvonplaten in https://github.com/huggingface/transformers/pull/13921
Fixes a minor doc issue (missing character) by @mishig25 in https://github.com/huggingface/transformers/pull/13922
Fix LED by @Rocketknight1 in https://github.com/huggingface/transformers/pull/13882
Add BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese by @datquocnguyen in https://github.com/huggingface/transformers/pull/13788
[trainer] memory metrics: add memory at the start report by @stas00 in https://github.com/huggingface/transformers/pull/13915
Image Segmentation pipeline by @mishig25 in https://github.com/huggingface/transformers/pull/13828
Adding support for tokens being suffixes or part of each other. by @Narsil in https://github.com/huggingface/transformers/pull/13918
Adds PreTrainedModel.framework attribute by @StellaAthena in https://github.com/huggingface/transformers/pull/13817
Fixed typo: herBERT -&gt; HerBERT by @adamjankaczmarek in https://github.com/huggingface/transformers/pull/13936
[Generation] Fix max_new_tokens by @patrickvonplaten in https://github.com/huggingface/transformers/pull/13919
Fix typo in README.md by @fullyz in https://github.com/huggingface/transformers/pull/13883
Update bug-report.md by @LysandreJik in https://github.com/huggingface/transformers/pull/13934
fix issue #13904 -attribute does not exist-  by @oraby8 in https://github.com/huggingface/transformers/pull/13942
Raise ValueError instead of asserts in src/transformers/benchmark/benchmark.py by @AkechiShiro in https://github.com/huggingface/transformers/pull/13951
Honor existing attention mask in tokenzier.pad by @sgugger in https://github.com/huggingface/transformers/pull/13926
[Gradient checkpoining] Correct disabling find_unused_parameters in Trainer when gradient checkpointing is enabled by @patrickvonplaten in https://github.com/huggingface/transformers/pull/13961
Change DataCollatorForSeq2Seq to pad labels to a multiple of pad_to_multiple_of by @affjljoo3581 in https://github.com/huggingface/transformers/pull/13949
Replace assert with unittest assertions by @LuisFerTR in https://github.com/huggingface/transformers/pull/13957
Raise exceptions instead of asserts in  src/transformers/data/processors/xnli.py by @midhun1998 in https://github.com/huggingface/transformers/pull/13945
Make username optional in hub_model_id by @sgugger in https://github.com/huggingface/transformers/pull/13940
Raise exceptions instead of asserts in src/transformers/data/processors/utils.py by @killazz67 in https://github.com/huggingface/transformers/pull/13938
Replace assert by ValueError of src/transformers/models/electra/modeling_{electra,tf_electra}.py and all other models that had copies by @AkechiShiro in https://github.com/huggingface/transformers/pull/13955
Fix missing tpu variable in benchmark_args_tf.py by @hardianlawi in https://github.com/huggingface/transformers/pull/13968
Specify im-seg mask greyscole mode by @mishig25 in https://github.com/huggingface/transformers/pull/13974
[Wav2Vec2] Make sure tensors are always bool for mask_indices by @patrickvonplaten in https://github.com/huggingface/transformers/pull/13977
Fixing the lecture values by making sure defaults are not changed by @Narsil in https://github.com/huggingface/transformers/pull/13976
[parallel doc] dealing with layers larger than one gpu by @stas00 in https://github.com/huggingface/transformers/pull/13980
Remove wrong model_args supplied by @qqaatw in https://github.com/huggingface/transformers/pull/13937
Allow single byte decoding by @patrickvonplaten in https://github.com/huggingface/transformers/pull/13988
Replace assertion with ValueError exception by @ddrm86 in https://github.com/huggingface/transformers/pull/14006
Add strong test for configuration attributes by @sgugger in https://github.com/huggingface/transformers/pull/14000
Fix FNet tokenizer tests by @LysandreJik in https://github.com/huggingface/transformers/pull/13995
[Testing] Move speech datasets to hf-internal testing ... by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14008
Raise exceptions instead of asserts in src/transformers/models/bart/modelingflax[bart, marian, mbart, pegasus].py by @killazz67 in https://github.com/huggingface/transformers/pull/13939
Scatter dummies + skip pipeline tests by @LysandreJik in https://github.com/huggingface/transformers/pull/13996
Fixed horizon_length for PPLM by @jacksukk in https://github.com/huggingface/transformers/pull/13886
Fix: replace assert statements with exceptions in file src/transformers/models/lxmert/modeling_lxmert.py by @murilo-goncalves in https://github.com/huggingface/transformers/pull/14029
[Docs] More general docstrings by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14028
[CLIP] minor fixes by @patil-suraj in https://github.com/huggingface/transformers/pull/14026
Don't duplicate the elements in dir by @sgugger in https://github.com/huggingface/transformers/pull/14023
Replace assertions with ValueError exceptions by @ddrm86 in https://github.com/huggingface/transformers/pull/14018
Fixes typo in modeling_speech_to_text by @mishig25 in https://github.com/huggingface/transformers/pull/14044
[Speech] Move all examples to new audio feature by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14045
Update SEW integration test tolerance by @anton-l in https://github.com/huggingface/transformers/pull/14048
[Flax] Clip fix test by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14046
Fix save when laod_best_model_at_end=True by @sgugger in https://github.com/huggingface/transformers/pull/14054
[Speech] Refactor Examples by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14040
fix typo by @yyy-Apple in https://github.com/huggingface/transformers/pull/14049
Fix typo by @ihoromi4 in https://github.com/huggingface/transformers/pull/14056
[FX] Fix passing None as concrete args when tracing by @thomasw21 in https://github.com/huggingface/transformers/pull/14022
TF Model train and eval step metrics for seq2seq models. by @pedro-r-marques in https://github.com/huggingface/transformers/pull/14009
update to_py_obj to support np.number by @PrettyMeng in https://github.com/huggingface/transformers/pull/14064
Trainer._load_rng_state() path fix (#14069) by @tlby in https://github.com/huggingface/transformers/pull/14071
replace assert with exception in src/transformers/utils/model_pararallel_utils.py by @skpig in https://github.com/huggingface/transformers/pull/14072
Add missing autocast() in Trainer.prediction_step() by @juice500ml in https://github.com/huggingface/transformers/pull/14075
Fix assert in src/transformers/data/datasets/language_modeling.py by @skpig in https://github.com/huggingface/transformers/pull/14077
Fix label attribution in token classification examples by @sgugger in https://github.com/huggingface/transformers/pull/14055
Context managers by @lvwerra in https://github.com/huggingface/transformers/pull/13900
Fix broken link in the translation section of task summaries by @h4iku in https://github.com/huggingface/transformers/pull/14087
[ASR] Small fix model card creation by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14093
Change asserts in src/transformers/models/xlnet/ to raise ValueError by @WestonKing-Leatham in https://github.com/huggingface/transformers/pull/14088
Replace assertions with ValueError exceptions by @ddrm86 in https://github.com/huggingface/transformers/pull/14061
[Typo] Replace "Masked" with "Causal" in TF CLM script by @cakiki in https://github.com/huggingface/transformers/pull/14014
[Examples] Add audio classification notebooks by @anton-l in https://github.com/huggingface/transformers/pull/14099
Fix ignore_mismatched_sizes by @qqaatw in https://github.com/huggingface/transformers/pull/14085
Fix typo in comment by @stalkermustang in https://github.com/huggingface/transformers/pull/14102
Replace assertion with ValueError exception by @ddrm86 in https://github.com/huggingface/transformers/pull/14098
fix typo in license docstring by @21jun in https://github.com/huggingface/transformers/pull/14094
Fix a typo in preprocessing docs by @h4iku in https://github.com/huggingface/transformers/pull/14108
Replace assertions with ValueError exceptions by @iDeepverma in https://github.com/huggingface/transformers/pull/14091
[tests] fix hubert test sort by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14116
Replace assert statements with exceptions (#13871) by @ddrm86 in https://github.com/huggingface/transformers/pull/13901
Translate README.md to Korean by @yeounyi in https://github.com/huggingface/transformers/pull/14015
Replace assertions with valueError Exeptions by @jyshdewangan in https://github.com/huggingface/transformers/pull/14117
Fix assertion in models by @skpig in https://github.com/huggingface/transformers/pull/14090
[wav2vec2] Add missing --validation_split_percentage data arg by @falcaopetri in https://github.com/huggingface/transformers/pull/14119
Rename variables with unclear naming by @qqaatw in https://github.com/huggingface/transformers/pull/14122
Update TP parallel GEMM image by @hyunwoongko in https://github.com/huggingface/transformers/pull/14112
Fix some typos in the docs by @h4iku in https://github.com/huggingface/transformers/pull/14126
Supporting Seq2Seq model for question answering task by @karthikrangasai in https://github.com/huggingface/transformers/pull/13432
Fix rendering of examples version links by @h4iku in https://github.com/huggingface/transformers/pull/14134
Fix some writing issues in the docs by @h4iku in https://github.com/huggingface/transformers/pull/14136
BartEnocder add set_input_embeddings by @Liangtaiwan in https://github.com/huggingface/transformers/pull/13960
Remove unneeded to_tensor() in TF inline example by @Rocketknight1 in https://github.com/huggingface/transformers/pull/14140
Enable DefaultDataCollator class by @Rocketknight1 in https://github.com/huggingface/transformers/pull/14141
Fix lazy init to stop hiding errors in import by @sgugger in https://github.com/huggingface/transformers/pull/14124
Add TF&lt;&gt;PT and Flax&lt;&gt;PT everywhere by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14047
Add Camembert to models exportable with ONNX by @ChainYo in https://github.com/huggingface/transformers/pull/14059
[Speech Recognition CTC] Add auth token to fine-tune private models by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14154
Add vision_encoder_decoder to models/init.py by @ydshieh in https://github.com/huggingface/transformers/pull/14151
[Speech Recognition] - Distributed training: Make sure vocab file removal and creation don't interfer  by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14161
Include Keras tensor in the allowed types by @sergiovalmac in https://github.com/huggingface/transformers/pull/14155
[megatron_gpt2] dynamic gelu, add tokenizer, save config by @stas00 in https://github.com/huggingface/transformers/pull/13928
Add Unispeech &amp; Unispeech-SAT by @patrickvonplaten in https://github.com/huggingface/transformers/pull/13963
[ONNX] Add symbolic function for XSoftmax op for exporting to ONNX. by @fatcat-z in https://github.com/huggingface/transformers/pull/14013
Typo on ner accelerate example code by @monologg in https://github.com/huggingface/transformers/pull/14150
fix typos in error messages in speech recognition example and modelcard.py by @mgoldey in https://github.com/huggingface/transformers/pull/14166
Replace assertions with ValueError exception by @huberemanuel in https://github.com/huggingface/transformers/pull/14142
switch to inference_mode from no_gard by @kamalkraj in https://github.com/huggingface/transformers/pull/13667
Fix gelu test for torch 1.10 by @LysandreJik in https://github.com/huggingface/transformers/pull/14167
[Gradient checkpointing] Enable for Deberta + DebertaV2 + SEW-D by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14175
[Pipelines] Fix ASR model types check by @anton-l in https://github.com/huggingface/transformers/pull/14178
Replace assert of data/data_collator.py by ValueError by @AkechiShiro in https://github.com/huggingface/transformers/pull/14131
[TPU tests] Enable first TPU examples pytorch by @patrickvonplaten in https://github.com/huggingface/transformers/pull/14121
[modeling_utils] respect original dtype in _get_resized_lm_head by @stas00 in https://github.com/huggingface/transformers/pull/14181

New Contributors

@arfon made their first contribution in https://github.com/huggingface/transformers/pull/13833
@silviu-oprea made their first contribution in https://github.com/huggingface/transformers/pull/13800
@yaserabdelaziz made their first contribution in https://github.com/huggingface/transformers/pull/13851
@Randl made their first contribution in https://github.com/huggingface/transformers/pull/13848
@Traubert made their first contribution in https://github.com/huggingface/transformers/pull/13757
@ZhaofengWu made their first contribution in https://github.com/huggingface/transformers/pull/13820
@m5l14i11 made their first contribution in https://github.com/huggingface/transformers/pull/13894
@hyunwoongko made their first contribution in https://github.com/huggingface/transformers/pull/13892
@ddrm86 made their first contribution in https://github.com/huggingface/transformers/pull/13871
@akulagrawal made their first contribution in https://github.com/huggingface/transformers/pull/13857
@yssjtu made their first contribution in https://github.com/huggingface/transformers/pull/13805
@ymwangg made their first contribution in https://github.com/huggingface/transformers/pull/13896
@hirotasoshu made their first contribution in https://github.com/huggingface/transformers/pull/13907
@fatcat-z made their first contribution in https://github.com/huggingface/transformers/pull/13765
@djroxx2000 made their first contribution in https://github.com/huggingface/transformers/pull/13909
@adamjankaczmarek made their first contribution in https://github.com/huggingface/transformers/pull/13936
@oraby8 made their first contribution in https://github.com/huggingface/transformers/pull/13942
@AkechiShiro made their first contribution in https://github.com/huggingface/transformers/pull/13951
@affjljoo3581 made their first contribution in https://github.com/huggingface/transformers/pull/13949
@LuisFerTR made their first contribution in https://github.com/huggingface/transformers/pull/13957
@midhun1998 made their first contribution in https://github.com/huggingface/transformers/pull/13945
@killazz67 made their first contribution in https://github.com/huggingface/transformers/pull/13938
@hardianlawi made their first contribution in https://github.com/huggingface/transformers/pull/13968
@jacksukk made their first contribution in https://github.com/huggingface/transformers/pull/13886
@murilo-goncalves made their first contribution in https://github.com/huggingface/transformers/pull/14029
@yyy-Apple made their first contribution in https://github.com/huggingface/transformers/pull/14049
@ihoromi4 made their first contribution in https://github.com/huggingface/transformers/pull/14056
@thomasw21 made their first contribution in https://github.com/huggingface/transformers/pull/14022
@pedro-r-marques made their first contribution in https://github.com/huggingface/transformers/pull/14009
@PrettyMeng made their first contribution in https://github.com/huggingface/transformers/pull/14064
@tlby made their first contribution in https://github.com/huggingface/transformers/pull/14071
@skpig made their first contribution in https://github.com/huggingface/transformers/pull/14072
@juice500ml made their first contribution in https://github.com/huggingface/transformers/pull/14075
@h4iku made their first contribution in https://github.com/huggingface/transformers/pull/14087
@WestonKing-Leatham made their first contribution in https://github.com/huggingface/transformers/pull/14088
@cakiki made their first contribution in https://github.com/huggingface/transformers/pull/14014
@stalkermustang made their first contribution in https://github.com/huggingface/transformers/pull/14102
@iDeepverma made their first contribution in https://github.com/huggingface/transformers/pull/14091
@yeounyi made their first contribution in https://github.com/huggingface/transformers/pull/14015
@jyshdewangan made their first contribution in https://github.com/huggingface/transformers/pull/14117
@karthikrangasai made their first contribution in https://github.com/huggingface/transformers/pull/13432
@ChainYo made their first contribution in https://github.com/huggingface/transformers/pull/14059
@sergiovalmac made their first contribution in https://github.com/huggingface/transformers/pull/14155
@huberemanuel made their first contribution in https://github.com/huggingface/transformers/pull/14142

Full Changelog: https://github.com/huggingface/transformers/compare/v4.11.0...v4.12.0</dc:description>
  <dc:description>If you use this software, please cite it using these metadata.</dc:description>
  <dc:identifier>https://zenodo.org/record/5608580</dc:identifier>
  <dc:identifier>10.5281/zenodo.5608580</dc:identifier>
  <dc:identifier>oai:zenodo.org:5608580</dc:identifier>
  <dc:relation>url:https://github.com/huggingface/transformers/tree/v4.12.0</dc:relation>
  <dc:relation>doi:10.5281/zenodo.3385997</dc:relation>
  <dc:relation>url:https://zenodo.org/communities/zenodo</dc:relation>
  <dc:rights>info:eu-repo/semantics/openAccess</dc:rights>
  <dc:title>Transformers: State-of-the-Art Natural Language Processing</dc:title>
  <dc:type>info:eu-repo/semantics/other</dc:type>
  <dc:type>software</dc:type>
</oai_dc:dc>
37,139
1,293
views
downloads
All versions This version
Views 37,139187
Downloads 1,2936
Data volume 10.0 GB73.7 MB
Unique views 30,889151
Unique downloads 6676

Share

Cite as