There is a newer version of this record available.

Software Open Access

Transformers: State-of-the-Art Natural Language Processing

Wolf, Thomas; Debut, Lysandre; Sanh, Victor; Chaumond, Julien; Delangue, Clement; Moi, Anthony; Cistac, Perric; Ma, Clara; Jernite, Yacine; Plu, Julien; Xu, Canwen; Le Scao, Teven; Gugger, Sylvain; Drame, Mariama; Lhoest, Quentin; Rush, Alexander M.


MARC21 XML Export

<?xml version='1.0' encoding='UTF-8'?>
<record xmlns="http://www.loc.gov/MARC21/slim">
  <leader>00000nmm##2200000uu#4500</leader>
  <controlfield tag="005">20211222193516.0</controlfield>
  <datafield tag="500" ind1=" " ind2=" ">
    <subfield code="a">If you use this software, please cite it using these metadata.</subfield>
  </datafield>
  <controlfield tag="001">5608580</controlfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Debut, Lysandre</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Sanh, Victor</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Chaumond, Julien</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Delangue, Clement</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Moi, Anthony</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Cistac, Perric</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Ma, Clara</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Jernite, Yacine</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Plu, Julien</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Xu, Canwen</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Le Scao, Teven</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Gugger, Sylvain</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Drame, Mariama</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Lhoest, Quentin</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Rush, Alexander M.</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">12281662</subfield>
    <subfield code="z">md5:5827cea086c67aaf85c8d73c0e4b2ab2</subfield>
    <subfield code="u">https://zenodo.org/record/5608580/files/huggingface/transformers-v4.12.0.zip</subfield>
  </datafield>
  <datafield tag="542" ind1=" " ind2=" ">
    <subfield code="l">open</subfield>
  </datafield>
  <datafield tag="260" ind1=" " ind2=" ">
    <subfield code="c">2020-10-01</subfield>
  </datafield>
  <datafield tag="909" ind1="C" ind2="O">
    <subfield code="p">software</subfield>
    <subfield code="p">user-zenodo</subfield>
    <subfield code="o">oai:zenodo.org:5608580</subfield>
  </datafield>
  <datafield tag="100" ind1=" " ind2=" ">
    <subfield code="a">Wolf, Thomas</subfield>
  </datafield>
  <datafield tag="245" ind1=" " ind2=" ">
    <subfield code="a">Transformers: State-of-the-Art Natural Language Processing</subfield>
  </datafield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">user-zenodo</subfield>
  </datafield>
  <datafield tag="540" ind1=" " ind2=" ">
    <subfield code="a">Other (Open)</subfield>
  </datafield>
  <datafield tag="650" ind1="1" ind2="7">
    <subfield code="a">cc-by</subfield>
    <subfield code="2">opendefinition.org</subfield>
  </datafield>
  <datafield tag="520" ind1=" " ind2=" ">
    <subfield code="a">TrOCR and VisionEncoderDecoderModel
&lt;p&gt;One new model is released as part of the TrOCR implementation: &lt;code&gt;TrOCRForCausalLM&lt;/code&gt;, in PyTorch. It comes along a new &lt;code&gt;VisionEncoderDecoderModel&lt;/code&gt; class, which allows to mix-and-match any vision Transformer encoder with any text Transformer as decoder, similar to the existing &lt;code&gt;SpeechEncoderDecoderModel&lt;/code&gt; class.&lt;/p&gt;
&lt;p&gt;The TrOCR model was proposed in &lt;a href="https://arxiv.org/abs/2109.10282"&gt;TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models&lt;/a&gt;, by Minghao Li, Tengchao Lv, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, Furu Wei.&lt;/p&gt;
&lt;p&gt;The TrOCR model consists of an image transformer encoder and an autoregressive text transformer to perform optical character recognition in an end-to-end manner.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Add TrOCR + VisionEncoderDecoderModel by @NielsRogge in &lt;a href="https://github.com/huggingface/transformers/pull/13874"&gt;https://github.com/huggingface/transformers/pull/13874&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Compatible checkpoints can be found on the Hub: &lt;a href="https://huggingface.co/models?other=trocr"&gt;https://huggingface.co/models?other=trocr&lt;/a&gt;&lt;/p&gt;
SEW &amp;amp; SEW-D
&lt;p&gt;SEW and SEW-D (Squeezed and Efficient Wav2Vec) were proposed in &lt;a href="https://arxiv.org/abs/2109.06870"&gt;Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition&lt;/a&gt; by Felix Wu, Kwangyoun Kim, Jing Pan, Kyu Han, Kilian Q. Weinberger, Yoav Artzi.&lt;/p&gt;
&lt;p&gt;SEW and SEW-D models use a Wav2Vec-style feature encoder and introduce temporal downsampling to reduce the length of the transformer encoder. SEW-D additionally replaces the transformer encoder with a DeBERTa one. Both models achieve significant inference speedups without sacrificing the speech recognition quality.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Add the SEW and SEW-D speech models by @anton-l in &lt;a href="https://github.com/huggingface/transformers/pull/13962"&gt;https://github.com/huggingface/transformers/pull/13962&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add SEW CTC models by @anton-l in &lt;a href="https://github.com/huggingface/transformers/pull/14158"&gt;https://github.com/huggingface/transformers/pull/14158&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Compatible checkpoints are available on the Hub: &lt;a href="https://huggingface.co/models?other=sew"&gt;https://huggingface.co/models?other=sew&lt;/a&gt; and &lt;a href="https://huggingface.co/models?other=sew-d"&gt;https://huggingface.co/models?other=sew-d&lt;/a&gt;&lt;/p&gt;
DistilHuBERT
&lt;p&gt;DistilHuBERT was proposed in &lt;a href="https://arxiv.org/abs/2110.01900"&gt;DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT&lt;/a&gt;, by Heng-Jui Chang, Shu-wen Yang, Hung-yi Lee.&lt;/p&gt;
&lt;p&gt;DistilHuBERT is a distilled version of the HuBERT model. Using only two transformer layers, the model scores competitively on the SUPERB benchmark tasks.&lt;/p&gt;
&lt;p&gt;Compatible checkpoint is available on the Hub: &lt;a href="https://huggingface.co/ntu-spml/distilhubert"&gt;https://huggingface.co/ntu-spml/distilhubert&lt;/a&gt;&lt;/p&gt;
TensorFlow improvements
&lt;p&gt;Several bug fixes and UX improvements for TensorFlow&lt;/p&gt;
Keras callback
&lt;p&gt;Introduction of a Keras callback to push to the hub each epoch, or after a given number of steps:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Keras callback to push to hub each epoch, or after N steps by @Rocketknight1 in &lt;a href="https://github.com/huggingface/transformers/pull/13773"&gt;https://github.com/huggingface/transformers/pull/13773&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
Updates on the encoder-decoder framework
&lt;p&gt;The encoder-decoder framework is now available in TensorFlow, allowing mixing and matching different encoders and decoders together into a single encoder-decoder architecture!&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Add TFEncoderDecoderModel + Add cross-attention to some TF models by @ydshieh in &lt;a href="https://github.com/huggingface/transformers/pull/13222"&gt;https://github.com/huggingface/transformers/pull/13222&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Besides this, the &lt;code&gt;EncoderDecoderModel&lt;/code&gt; classes have been updated to work similar to models like BART and T5. From now on, users don't need to pass &lt;code&gt;decoder_input_ids&lt;/code&gt; themselves anymore to the model. Instead, they will be created automatically based on the &lt;code&gt;labels&lt;/code&gt; (namely by shifting them one position to the right, replacing -100 by the &lt;code&gt;pad_token_id&lt;/code&gt; and prepending the &lt;code&gt;decoder_start_token_id&lt;/code&gt;). Note that this may result in training discrepancies if fine-tuning a model trained with versions anterior to 4.12.0 that set the &lt;code&gt;decoder_input_ids&lt;/code&gt; = &lt;code&gt;labels&lt;/code&gt;.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Fix EncoderDecoderModel classes to be more like BART and T5 by @NielsRogge  in &lt;a href="https://github.com/huggingface/transformers/pull/14139"&gt;https://github.com/huggingface/transformers/pull/14139&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
Speech improvements
&lt;ul&gt;
&lt;li&gt;Add DistilHuBERT  by @anton-l in &lt;a href="https://github.com/huggingface/transformers/pull/14174"&gt;https://github.com/huggingface/transformers/pull/14174&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Speech Examples] Add pytorch speech pretraining by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/13877"&gt;https://github.com/huggingface/transformers/pull/13877&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Speech Examples] Add new audio feature by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14027"&gt;https://github.com/huggingface/transformers/pull/14027&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add ASR colabs by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14067"&gt;https://github.com/huggingface/transformers/pull/14067&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[ASR] Make speech recognition example more general to load any tokenizer by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14079"&gt;https://github.com/huggingface/transformers/pull/14079&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Examples] Add an official audio classification example by @anton-l in &lt;a href="https://github.com/huggingface/transformers/pull/13722"&gt;https://github.com/huggingface/transformers/pull/13722&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Examples] Use Audio feature in speech classification by @anton-l in &lt;a href="https://github.com/huggingface/transformers/pull/14052"&gt;https://github.com/huggingface/transformers/pull/14052&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
Auto-model API
&lt;p&gt;To make it easier to extend the Transformers library, every Auto class a new &lt;code&gt;register&lt;/code&gt; method, that allows you to register your own custom models, configurations or tokenizers. See more in the &lt;a href="https://huggingface.co/transformers/model_doc/auto.html#extending-the-auto-classes"&gt;documentation&lt;/a&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Add an API to register objects to Auto classes by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/13989"&gt;https://github.com/huggingface/transformers/pull/13989&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
Bug fixes and improvements
&lt;ul&gt;
&lt;li&gt;Fix filtering in test fetcher utils by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/13766"&gt;https://github.com/huggingface/transformers/pull/13766&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix warning for gradient_checkpointing by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/13767"&gt;https://github.com/huggingface/transformers/pull/13767&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Implement len in IterableDatasetShard by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/13780"&gt;https://github.com/huggingface/transformers/pull/13780&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Wav2Vec2] Better error message by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/13777"&gt;https://github.com/huggingface/transformers/pull/13777&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix LayoutLM ONNX test error by @nishprabhu in &lt;a href="https://github.com/huggingface/transformers/pull/13710"&gt;https://github.com/huggingface/transformers/pull/13710&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Enable readme link synchronization by @qqaatw in &lt;a href="https://github.com/huggingface/transformers/pull/13785"&gt;https://github.com/huggingface/transformers/pull/13785&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix length of IterableDatasetShard and add test by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/13792"&gt;https://github.com/huggingface/transformers/pull/13792&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[docs/gpt-j] addd instructions for how minimize CPU RAM usage by @patil-suraj in &lt;a href="https://github.com/huggingface/transformers/pull/13795"&gt;https://github.com/huggingface/transformers/pull/13795&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[examples &lt;code&gt;run_glue.py&lt;/code&gt;] missing requirements &lt;code&gt;scipy&lt;/code&gt;, &lt;code&gt;sklearn&lt;/code&gt; by @stas00 in &lt;a href="https://github.com/huggingface/transformers/pull/13768"&gt;https://github.com/huggingface/transformers/pull/13768&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[examples/flax] use Repository API for push_to_hub by @patil-suraj in &lt;a href="https://github.com/huggingface/transformers/pull/13672"&gt;https://github.com/huggingface/transformers/pull/13672&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix gather for TPU by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/13813"&gt;https://github.com/huggingface/transformers/pull/13813&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[testing] auto-replay captured streams by @stas00 in &lt;a href="https://github.com/huggingface/transformers/pull/13803"&gt;https://github.com/huggingface/transformers/pull/13803&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add MultiBERTs conversion script by @gchhablani in &lt;a href="https://github.com/huggingface/transformers/pull/13077"&gt;https://github.com/huggingface/transformers/pull/13077&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Examples] Improve mapping in accelerate examples by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/13810"&gt;https://github.com/huggingface/transformers/pull/13810&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[DPR] Correct init by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/13796"&gt;https://github.com/huggingface/transformers/pull/13796&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;skip gptj slow generate tests by @patil-suraj in &lt;a href="https://github.com/huggingface/transformers/pull/13809"&gt;https://github.com/huggingface/transformers/pull/13809&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix warning situation: UserWarning: max_length is ignored when padding=True" by @shirayu in &lt;a href="https://github.com/huggingface/transformers/pull/13829"&gt;https://github.com/huggingface/transformers/pull/13829&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Updating CITATION.cff to fix GitHub citation prompt BibTeX output. by @arfon in &lt;a href="https://github.com/huggingface/transformers/pull/13833"&gt;https://github.com/huggingface/transformers/pull/13833&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add TF notebooks by @Rocketknight1 in &lt;a href="https://github.com/huggingface/transformers/pull/13793"&gt;https://github.com/huggingface/transformers/pull/13793&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Bart: check if decoder_inputs_embeds is set by @silviu-oprea in &lt;a href="https://github.com/huggingface/transformers/pull/13800"&gt;https://github.com/huggingface/transformers/pull/13800&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;include megatron_gpt2 in installed modules by @stas00 in &lt;a href="https://github.com/huggingface/transformers/pull/13834"&gt;https://github.com/huggingface/transformers/pull/13834&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Delete MultiBERTs conversion script by @gchhablani in &lt;a href="https://github.com/huggingface/transformers/pull/13852"&gt;https://github.com/huggingface/transformers/pull/13852&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Remove a duplicated bullet point in the GPT-J doc by @yaserabdelaziz in &lt;a href="https://github.com/huggingface/transformers/pull/13851"&gt;https://github.com/huggingface/transformers/pull/13851&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add Mistral GPT-2 Stability Tweaks by @siddk in &lt;a href="https://github.com/huggingface/transformers/pull/13573"&gt;https://github.com/huggingface/transformers/pull/13573&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix broken link to distill models in docs by @Randl in &lt;a href="https://github.com/huggingface/transformers/pull/13848"&gt;https://github.com/huggingface/transformers/pull/13848&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;:sparkles: update image classification example by @nateraw in &lt;a href="https://github.com/huggingface/transformers/pull/13824"&gt;https://github.com/huggingface/transformers/pull/13824&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Update no_* argument (HfArgumentParser) by @BramVanroy in &lt;a href="https://github.com/huggingface/transformers/pull/13865"&gt;https://github.com/huggingface/transformers/pull/13865&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Update Tatoeba conversion by @Traubert in &lt;a href="https://github.com/huggingface/transformers/pull/13757"&gt;https://github.com/huggingface/transformers/pull/13757&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fixing 1-length special tokens cut. by @Narsil in &lt;a href="https://github.com/huggingface/transformers/pull/13862"&gt;https://github.com/huggingface/transformers/pull/13862&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix flax summarization example: save checkpoint after each epoch and push checkpoint to the hub by @ydshieh in &lt;a href="https://github.com/huggingface/transformers/pull/13872"&gt;https://github.com/huggingface/transformers/pull/13872&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fixing empty prompts for text-generation when BOS exists. by @Narsil in &lt;a href="https://github.com/huggingface/transformers/pull/13859"&gt;https://github.com/huggingface/transformers/pull/13859&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Improve error message when loading models from Hub by @aphedges in &lt;a href="https://github.com/huggingface/transformers/pull/13836"&gt;https://github.com/huggingface/transformers/pull/13836&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Initial support for symbolic tracing with torch.fx allowing dynamic axes by @michaelbenayoun in &lt;a href="https://github.com/huggingface/transformers/pull/13579"&gt;https://github.com/huggingface/transformers/pull/13579&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Allow dataset to be an optional argument for (Distributed)LengthGroupedSampler by @ZhaofengWu in &lt;a href="https://github.com/huggingface/transformers/pull/13820"&gt;https://github.com/huggingface/transformers/pull/13820&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fixing question-answering with long contexts  by @Narsil in &lt;a href="https://github.com/huggingface/transformers/pull/13873"&gt;https://github.com/huggingface/transformers/pull/13873&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;fix(integrations): consider test metrics by @borisdayma in &lt;a href="https://github.com/huggingface/transformers/pull/13888"&gt;https://github.com/huggingface/transformers/pull/13888&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;fix: replace asserts by value error by @m5l14i11 in &lt;a href="https://github.com/huggingface/transformers/pull/13894"&gt;https://github.com/huggingface/transformers/pull/13894&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Update parallelism.md by @hyunwoongko in &lt;a href="https://github.com/huggingface/transformers/pull/13892"&gt;https://github.com/huggingface/transformers/pull/13892&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Autodocument the list of ONNX-supported models by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/13884"&gt;https://github.com/huggingface/transformers/pull/13884&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fixing GPU for token-classification in a better way. by @Narsil in &lt;a href="https://github.com/huggingface/transformers/pull/13856"&gt;https://github.com/huggingface/transformers/pull/13856&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Update FSNER code in examples-&amp;gt;research_projects-&amp;gt;fsner by @sayef in &lt;a href="https://github.com/huggingface/transformers/pull/13864"&gt;https://github.com/huggingface/transformers/pull/13864&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assert statements with exceptions by @ddrm86 in &lt;a href="https://github.com/huggingface/transformers/pull/13871"&gt;https://github.com/huggingface/transformers/pull/13871&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fixing Backward compatiblity for zero-shot by @Narsil in &lt;a href="https://github.com/huggingface/transformers/pull/13855"&gt;https://github.com/huggingface/transformers/pull/13855&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Update run_qa.py - CorrectTypo by @akulagrawal in &lt;a href="https://github.com/huggingface/transformers/pull/13857"&gt;https://github.com/huggingface/transformers/pull/13857&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;T5ForConditionalGeneration: enabling using past_key_values and labels in training by @yssjtu in &lt;a href="https://github.com/huggingface/transformers/pull/13805"&gt;https://github.com/huggingface/transformers/pull/13805&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix trainer logging_nan_inf_filter in torch_xla mode by @ymwangg in &lt;a href="https://github.com/huggingface/transformers/pull/13896"&gt;https://github.com/huggingface/transformers/pull/13896&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix hp search for non sigopt backends by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/13897"&gt;https://github.com/huggingface/transformers/pull/13897&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Trainer] Fix nan-loss condition by @anton-l in &lt;a href="https://github.com/huggingface/transformers/pull/13911"&gt;https://github.com/huggingface/transformers/pull/13911&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Raise exceptions instead of asserts in utils/download_glue_data by @hirotasoshu in &lt;a href="https://github.com/huggingface/transformers/pull/13907"&gt;https://github.com/huggingface/transformers/pull/13907&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add an example of exporting BartModel + BeamSearch to ONNX module. by @fatcat-z in &lt;a href="https://github.com/huggingface/transformers/pull/13765"&gt;https://github.com/huggingface/transformers/pull/13765&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;#12789 Replace assert statements with exceptions by @djroxx2000 in &lt;a href="https://github.com/huggingface/transformers/pull/13909"&gt;https://github.com/huggingface/transformers/pull/13909&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add missing whitespace to multiline strings by @aphedges in &lt;a href="https://github.com/huggingface/transformers/pull/13916"&gt;https://github.com/huggingface/transformers/pull/13916&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Wav2Vec2] Fix mask_feature_prob by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/13921"&gt;https://github.com/huggingface/transformers/pull/13921&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fixes a minor doc issue (missing character) by @mishig25 in &lt;a href="https://github.com/huggingface/transformers/pull/13922"&gt;https://github.com/huggingface/transformers/pull/13922&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix LED by @Rocketknight1 in &lt;a href="https://github.com/huggingface/transformers/pull/13882"&gt;https://github.com/huggingface/transformers/pull/13882&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese by @datquocnguyen in &lt;a href="https://github.com/huggingface/transformers/pull/13788"&gt;https://github.com/huggingface/transformers/pull/13788&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[trainer] memory metrics: add memory at the start report by @stas00 in &lt;a href="https://github.com/huggingface/transformers/pull/13915"&gt;https://github.com/huggingface/transformers/pull/13915&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Image Segmentation pipeline by @mishig25 in &lt;a href="https://github.com/huggingface/transformers/pull/13828"&gt;https://github.com/huggingface/transformers/pull/13828&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Adding support for tokens being suffixes or part of each other. by @Narsil in &lt;a href="https://github.com/huggingface/transformers/pull/13918"&gt;https://github.com/huggingface/transformers/pull/13918&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Adds &lt;code&gt;PreTrainedModel.framework&lt;/code&gt; attribute by @StellaAthena in &lt;a href="https://github.com/huggingface/transformers/pull/13817"&gt;https://github.com/huggingface/transformers/pull/13817&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fixed typo: herBERT -&amp;gt; HerBERT by @adamjankaczmarek in &lt;a href="https://github.com/huggingface/transformers/pull/13936"&gt;https://github.com/huggingface/transformers/pull/13936&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Generation] Fix max_new_tokens by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/13919"&gt;https://github.com/huggingface/transformers/pull/13919&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix typo in README.md by @fullyz in &lt;a href="https://github.com/huggingface/transformers/pull/13883"&gt;https://github.com/huggingface/transformers/pull/13883&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Update bug-report.md by @LysandreJik in &lt;a href="https://github.com/huggingface/transformers/pull/13934"&gt;https://github.com/huggingface/transformers/pull/13934&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;fix issue #13904 -attribute does not exist-  by @oraby8 in &lt;a href="https://github.com/huggingface/transformers/pull/13942"&gt;https://github.com/huggingface/transformers/pull/13942&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Raise ValueError instead of asserts in src/transformers/benchmark/benchmark.py by @AkechiShiro in &lt;a href="https://github.com/huggingface/transformers/pull/13951"&gt;https://github.com/huggingface/transformers/pull/13951&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Honor existing attention mask in tokenzier.pad by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/13926"&gt;https://github.com/huggingface/transformers/pull/13926&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Gradient checkpoining] Correct disabling &lt;code&gt;find_unused_parameters&lt;/code&gt; in Trainer when gradient checkpointing is enabled by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/13961"&gt;https://github.com/huggingface/transformers/pull/13961&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Change DataCollatorForSeq2Seq to pad labels to a multiple of &lt;code&gt;pad_to_multiple_of&lt;/code&gt; by @affjljoo3581 in &lt;a href="https://github.com/huggingface/transformers/pull/13949"&gt;https://github.com/huggingface/transformers/pull/13949&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assert with unittest assertions by @LuisFerTR in &lt;a href="https://github.com/huggingface/transformers/pull/13957"&gt;https://github.com/huggingface/transformers/pull/13957&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Raise exceptions instead of asserts in  src/transformers/data/processors/xnli.py by @midhun1998 in &lt;a href="https://github.com/huggingface/transformers/pull/13945"&gt;https://github.com/huggingface/transformers/pull/13945&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Make username optional in hub_model_id by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/13940"&gt;https://github.com/huggingface/transformers/pull/13940&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Raise exceptions instead of asserts in src/transformers/data/processors/utils.py by @killazz67 in &lt;a href="https://github.com/huggingface/transformers/pull/13938"&gt;https://github.com/huggingface/transformers/pull/13938&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assert by ValueError of src/transformers/models/electra/modeling_{electra,tf_electra}.py and all other models that had copies by @AkechiShiro in &lt;a href="https://github.com/huggingface/transformers/pull/13955"&gt;https://github.com/huggingface/transformers/pull/13955&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix missing tpu variable in benchmark_args_tf.py by @hardianlawi in &lt;a href="https://github.com/huggingface/transformers/pull/13968"&gt;https://github.com/huggingface/transformers/pull/13968&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Specify im-seg mask greyscole mode by @mishig25 in &lt;a href="https://github.com/huggingface/transformers/pull/13974"&gt;https://github.com/huggingface/transformers/pull/13974&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Wav2Vec2] Make sure tensors are always bool for mask_indices by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/13977"&gt;https://github.com/huggingface/transformers/pull/13977&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fixing the lecture values by making sure defaults are not changed by @Narsil in &lt;a href="https://github.com/huggingface/transformers/pull/13976"&gt;https://github.com/huggingface/transformers/pull/13976&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[parallel doc] dealing with layers larger than one gpu by @stas00 in &lt;a href="https://github.com/huggingface/transformers/pull/13980"&gt;https://github.com/huggingface/transformers/pull/13980&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Remove wrong model_args supplied by @qqaatw in &lt;a href="https://github.com/huggingface/transformers/pull/13937"&gt;https://github.com/huggingface/transformers/pull/13937&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Allow single byte decoding by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/13988"&gt;https://github.com/huggingface/transformers/pull/13988&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assertion with ValueError exception by @ddrm86 in &lt;a href="https://github.com/huggingface/transformers/pull/14006"&gt;https://github.com/huggingface/transformers/pull/14006&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add strong test for configuration attributes by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/14000"&gt;https://github.com/huggingface/transformers/pull/14000&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix FNet tokenizer tests by @LysandreJik in &lt;a href="https://github.com/huggingface/transformers/pull/13995"&gt;https://github.com/huggingface/transformers/pull/13995&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Testing] Move speech datasets to &lt;code&gt;hf-internal&lt;/code&gt; testing ... by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14008"&gt;https://github.com/huggingface/transformers/pull/14008&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Raise exceptions instead of asserts in src/transformers/models/bart/modeling&lt;em&gt;flax&lt;/em&gt;[bart, marian, mbart, pegasus].py by @killazz67 in &lt;a href="https://github.com/huggingface/transformers/pull/13939"&gt;https://github.com/huggingface/transformers/pull/13939&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Scatter dummies + skip pipeline tests by @LysandreJik in &lt;a href="https://github.com/huggingface/transformers/pull/13996"&gt;https://github.com/huggingface/transformers/pull/13996&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fixed horizon_length for PPLM by @jacksukk in &lt;a href="https://github.com/huggingface/transformers/pull/13886"&gt;https://github.com/huggingface/transformers/pull/13886&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix: replace assert statements with exceptions in file src/transformers/models/lxmert/modeling_lxmert.py by @murilo-goncalves in &lt;a href="https://github.com/huggingface/transformers/pull/14029"&gt;https://github.com/huggingface/transformers/pull/14029&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Docs] More general docstrings by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14028"&gt;https://github.com/huggingface/transformers/pull/14028&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[CLIP] minor fixes by @patil-suraj in &lt;a href="https://github.com/huggingface/transformers/pull/14026"&gt;https://github.com/huggingface/transformers/pull/14026&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Don't duplicate the elements in dir by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/14023"&gt;https://github.com/huggingface/transformers/pull/14023&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assertions with ValueError exceptions by @ddrm86 in &lt;a href="https://github.com/huggingface/transformers/pull/14018"&gt;https://github.com/huggingface/transformers/pull/14018&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fixes typo in &lt;code&gt;modeling_speech_to_text&lt;/code&gt; by @mishig25 in &lt;a href="https://github.com/huggingface/transformers/pull/14044"&gt;https://github.com/huggingface/transformers/pull/14044&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Speech] Move all examples to new audio feature by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14045"&gt;https://github.com/huggingface/transformers/pull/14045&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Update SEW integration test tolerance by @anton-l in &lt;a href="https://github.com/huggingface/transformers/pull/14048"&gt;https://github.com/huggingface/transformers/pull/14048&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Flax] Clip fix test by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14046"&gt;https://github.com/huggingface/transformers/pull/14046&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix save when laod_best_model_at_end=True by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/14054"&gt;https://github.com/huggingface/transformers/pull/14054&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Speech] Refactor Examples by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14040"&gt;https://github.com/huggingface/transformers/pull/14040&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;fix typo by @yyy-Apple in &lt;a href="https://github.com/huggingface/transformers/pull/14049"&gt;https://github.com/huggingface/transformers/pull/14049&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix typo by @ihoromi4 in &lt;a href="https://github.com/huggingface/transformers/pull/14056"&gt;https://github.com/huggingface/transformers/pull/14056&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[FX] Fix passing None as concrete args when tracing by @thomasw21 in &lt;a href="https://github.com/huggingface/transformers/pull/14022"&gt;https://github.com/huggingface/transformers/pull/14022&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;TF Model train and eval step metrics for seq2seq models. by @pedro-r-marques in &lt;a href="https://github.com/huggingface/transformers/pull/14009"&gt;https://github.com/huggingface/transformers/pull/14009&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;update to_py_obj to support np.number by @PrettyMeng in &lt;a href="https://github.com/huggingface/transformers/pull/14064"&gt;https://github.com/huggingface/transformers/pull/14064&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Trainer._load_rng_state() path fix (#14069) by @tlby in &lt;a href="https://github.com/huggingface/transformers/pull/14071"&gt;https://github.com/huggingface/transformers/pull/14071&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;replace assert with exception in src/transformers/utils/model_pararallel_utils.py by @skpig in &lt;a href="https://github.com/huggingface/transformers/pull/14072"&gt;https://github.com/huggingface/transformers/pull/14072&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add missing autocast() in Trainer.prediction_step() by @juice500ml in &lt;a href="https://github.com/huggingface/transformers/pull/14075"&gt;https://github.com/huggingface/transformers/pull/14075&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix assert in src/transformers/data/datasets/language_modeling.py by @skpig in &lt;a href="https://github.com/huggingface/transformers/pull/14077"&gt;https://github.com/huggingface/transformers/pull/14077&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix label attribution in token classification examples by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/14055"&gt;https://github.com/huggingface/transformers/pull/14055&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Context managers by @lvwerra in &lt;a href="https://github.com/huggingface/transformers/pull/13900"&gt;https://github.com/huggingface/transformers/pull/13900&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix broken link in the translation section of task summaries by @h4iku in &lt;a href="https://github.com/huggingface/transformers/pull/14087"&gt;https://github.com/huggingface/transformers/pull/14087&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[ASR] Small fix model card creation by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14093"&gt;https://github.com/huggingface/transformers/pull/14093&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Change asserts in src/transformers/models/xlnet/ to raise ValueError by @WestonKing-Leatham in &lt;a href="https://github.com/huggingface/transformers/pull/14088"&gt;https://github.com/huggingface/transformers/pull/14088&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assertions with ValueError exceptions by @ddrm86 in &lt;a href="https://github.com/huggingface/transformers/pull/14061"&gt;https://github.com/huggingface/transformers/pull/14061&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Typo] Replace "Masked" with "Causal" in TF CLM script by @cakiki in &lt;a href="https://github.com/huggingface/transformers/pull/14014"&gt;https://github.com/huggingface/transformers/pull/14014&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Examples] Add audio classification notebooks by @anton-l in &lt;a href="https://github.com/huggingface/transformers/pull/14099"&gt;https://github.com/huggingface/transformers/pull/14099&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix ignore_mismatched_sizes by @qqaatw in &lt;a href="https://github.com/huggingface/transformers/pull/14085"&gt;https://github.com/huggingface/transformers/pull/14085&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix typo in comment by @stalkermustang in &lt;a href="https://github.com/huggingface/transformers/pull/14102"&gt;https://github.com/huggingface/transformers/pull/14102&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assertion with ValueError exception by @ddrm86 in &lt;a href="https://github.com/huggingface/transformers/pull/14098"&gt;https://github.com/huggingface/transformers/pull/14098&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;fix typo in license docstring by @21jun in &lt;a href="https://github.com/huggingface/transformers/pull/14094"&gt;https://github.com/huggingface/transformers/pull/14094&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix a typo in preprocessing docs by @h4iku in &lt;a href="https://github.com/huggingface/transformers/pull/14108"&gt;https://github.com/huggingface/transformers/pull/14108&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assertions with ValueError exceptions by @iDeepverma in &lt;a href="https://github.com/huggingface/transformers/pull/14091"&gt;https://github.com/huggingface/transformers/pull/14091&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[tests] fix hubert test sort by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14116"&gt;https://github.com/huggingface/transformers/pull/14116&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assert statements with exceptions (#13871) by @ddrm86 in &lt;a href="https://github.com/huggingface/transformers/pull/13901"&gt;https://github.com/huggingface/transformers/pull/13901&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Translate README.md to Korean by @yeounyi in &lt;a href="https://github.com/huggingface/transformers/pull/14015"&gt;https://github.com/huggingface/transformers/pull/14015&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assertions with valueError Exeptions by @jyshdewangan in &lt;a href="https://github.com/huggingface/transformers/pull/14117"&gt;https://github.com/huggingface/transformers/pull/14117&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix assertion in models by @skpig in &lt;a href="https://github.com/huggingface/transformers/pull/14090"&gt;https://github.com/huggingface/transformers/pull/14090&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[wav2vec2] Add missing --validation_split_percentage data arg by @falcaopetri in &lt;a href="https://github.com/huggingface/transformers/pull/14119"&gt;https://github.com/huggingface/transformers/pull/14119&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Rename variables with unclear naming by @qqaatw in &lt;a href="https://github.com/huggingface/transformers/pull/14122"&gt;https://github.com/huggingface/transformers/pull/14122&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Update TP parallel GEMM image by @hyunwoongko in &lt;a href="https://github.com/huggingface/transformers/pull/14112"&gt;https://github.com/huggingface/transformers/pull/14112&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix some typos in the docs by @h4iku in &lt;a href="https://github.com/huggingface/transformers/pull/14126"&gt;https://github.com/huggingface/transformers/pull/14126&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Supporting Seq2Seq model for question answering task by @karthikrangasai in &lt;a href="https://github.com/huggingface/transformers/pull/13432"&gt;https://github.com/huggingface/transformers/pull/13432&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix rendering of examples version links by @h4iku in &lt;a href="https://github.com/huggingface/transformers/pull/14134"&gt;https://github.com/huggingface/transformers/pull/14134&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix some writing issues in the docs by @h4iku in &lt;a href="https://github.com/huggingface/transformers/pull/14136"&gt;https://github.com/huggingface/transformers/pull/14136&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;BartEnocder add set_input_embeddings by @Liangtaiwan in &lt;a href="https://github.com/huggingface/transformers/pull/13960"&gt;https://github.com/huggingface/transformers/pull/13960&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Remove unneeded &lt;code&gt;to_tensor()&lt;/code&gt; in TF inline example by @Rocketknight1 in &lt;a href="https://github.com/huggingface/transformers/pull/14140"&gt;https://github.com/huggingface/transformers/pull/14140&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Enable DefaultDataCollator class by @Rocketknight1 in &lt;a href="https://github.com/huggingface/transformers/pull/14141"&gt;https://github.com/huggingface/transformers/pull/14141&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix lazy init to stop hiding errors in import by @sgugger in &lt;a href="https://github.com/huggingface/transformers/pull/14124"&gt;https://github.com/huggingface/transformers/pull/14124&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add TF&amp;lt;&amp;gt;PT and Flax&amp;lt;&amp;gt;PT everywhere by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14047"&gt;https://github.com/huggingface/transformers/pull/14047&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add Camembert to models exportable with ONNX by @ChainYo in &lt;a href="https://github.com/huggingface/transformers/pull/14059"&gt;https://github.com/huggingface/transformers/pull/14059&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Speech Recognition CTC] Add auth token to fine-tune private models by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14154"&gt;https://github.com/huggingface/transformers/pull/14154&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add vision_encoder_decoder to models/&lt;strong&gt;init&lt;/strong&gt;.py by @ydshieh in &lt;a href="https://github.com/huggingface/transformers/pull/14151"&gt;https://github.com/huggingface/transformers/pull/14151&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Speech Recognition] - Distributed training: Make sure vocab file removal and creation don't interfer  by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14161"&gt;https://github.com/huggingface/transformers/pull/14161&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Include Keras tensor in the allowed types by @sergiovalmac in &lt;a href="https://github.com/huggingface/transformers/pull/14155"&gt;https://github.com/huggingface/transformers/pull/14155&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[megatron_gpt2] dynamic gelu, add tokenizer, save config by @stas00 in &lt;a href="https://github.com/huggingface/transformers/pull/13928"&gt;https://github.com/huggingface/transformers/pull/13928&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Add Unispeech &amp;amp; Unispeech-SAT by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/13963"&gt;https://github.com/huggingface/transformers/pull/13963&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[ONNX] Add symbolic function for XSoftmax op for exporting to ONNX. by @fatcat-z in &lt;a href="https://github.com/huggingface/transformers/pull/14013"&gt;https://github.com/huggingface/transformers/pull/14013&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Typo on ner accelerate example code by @monologg in &lt;a href="https://github.com/huggingface/transformers/pull/14150"&gt;https://github.com/huggingface/transformers/pull/14150&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;fix typos in error messages in speech recognition example and modelcard.py by @mgoldey in &lt;a href="https://github.com/huggingface/transformers/pull/14166"&gt;https://github.com/huggingface/transformers/pull/14166&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assertions with ValueError exception by @huberemanuel in &lt;a href="https://github.com/huggingface/transformers/pull/14142"&gt;https://github.com/huggingface/transformers/pull/14142&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;switch to inference_mode from no_gard by @kamalkraj in &lt;a href="https://github.com/huggingface/transformers/pull/13667"&gt;https://github.com/huggingface/transformers/pull/13667&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Fix gelu test for torch 1.10 by @LysandreJik in &lt;a href="https://github.com/huggingface/transformers/pull/14167"&gt;https://github.com/huggingface/transformers/pull/14167&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Gradient checkpointing] Enable for Deberta + DebertaV2 + SEW-D by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14175"&gt;https://github.com/huggingface/transformers/pull/14175&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[Pipelines] Fix ASR model types check by @anton-l in &lt;a href="https://github.com/huggingface/transformers/pull/14178"&gt;https://github.com/huggingface/transformers/pull/14178&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Replace assert of data/data_collator.py by ValueError by @AkechiShiro in &lt;a href="https://github.com/huggingface/transformers/pull/14131"&gt;https://github.com/huggingface/transformers/pull/14131&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[TPU tests] Enable first TPU examples pytorch by @patrickvonplaten in &lt;a href="https://github.com/huggingface/transformers/pull/14121"&gt;https://github.com/huggingface/transformers/pull/14121&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;[modeling_utils] respect original dtype in _get_resized_lm_head by @stas00 in &lt;a href="https://github.com/huggingface/transformers/pull/14181"&gt;https://github.com/huggingface/transformers/pull/14181&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
New Contributors
&lt;ul&gt;
&lt;li&gt;@arfon made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13833"&gt;https://github.com/huggingface/transformers/pull/13833&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@silviu-oprea made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13800"&gt;https://github.com/huggingface/transformers/pull/13800&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@yaserabdelaziz made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13851"&gt;https://github.com/huggingface/transformers/pull/13851&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@Randl made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13848"&gt;https://github.com/huggingface/transformers/pull/13848&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@Traubert made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13757"&gt;https://github.com/huggingface/transformers/pull/13757&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@ZhaofengWu made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13820"&gt;https://github.com/huggingface/transformers/pull/13820&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@m5l14i11 made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13894"&gt;https://github.com/huggingface/transformers/pull/13894&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@hyunwoongko made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13892"&gt;https://github.com/huggingface/transformers/pull/13892&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@ddrm86 made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13871"&gt;https://github.com/huggingface/transformers/pull/13871&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@akulagrawal made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13857"&gt;https://github.com/huggingface/transformers/pull/13857&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@yssjtu made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13805"&gt;https://github.com/huggingface/transformers/pull/13805&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@ymwangg made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13896"&gt;https://github.com/huggingface/transformers/pull/13896&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@hirotasoshu made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13907"&gt;https://github.com/huggingface/transformers/pull/13907&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@fatcat-z made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13765"&gt;https://github.com/huggingface/transformers/pull/13765&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@djroxx2000 made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13909"&gt;https://github.com/huggingface/transformers/pull/13909&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@adamjankaczmarek made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13936"&gt;https://github.com/huggingface/transformers/pull/13936&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@oraby8 made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13942"&gt;https://github.com/huggingface/transformers/pull/13942&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@AkechiShiro made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13951"&gt;https://github.com/huggingface/transformers/pull/13951&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@affjljoo3581 made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13949"&gt;https://github.com/huggingface/transformers/pull/13949&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@LuisFerTR made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13957"&gt;https://github.com/huggingface/transformers/pull/13957&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@midhun1998 made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13945"&gt;https://github.com/huggingface/transformers/pull/13945&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@killazz67 made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13938"&gt;https://github.com/huggingface/transformers/pull/13938&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@hardianlawi made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13968"&gt;https://github.com/huggingface/transformers/pull/13968&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@jacksukk made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13886"&gt;https://github.com/huggingface/transformers/pull/13886&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@murilo-goncalves made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14029"&gt;https://github.com/huggingface/transformers/pull/14029&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@yyy-Apple made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14049"&gt;https://github.com/huggingface/transformers/pull/14049&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@ihoromi4 made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14056"&gt;https://github.com/huggingface/transformers/pull/14056&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@thomasw21 made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14022"&gt;https://github.com/huggingface/transformers/pull/14022&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@pedro-r-marques made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14009"&gt;https://github.com/huggingface/transformers/pull/14009&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@PrettyMeng made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14064"&gt;https://github.com/huggingface/transformers/pull/14064&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@tlby made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14071"&gt;https://github.com/huggingface/transformers/pull/14071&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@skpig made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14072"&gt;https://github.com/huggingface/transformers/pull/14072&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@juice500ml made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14075"&gt;https://github.com/huggingface/transformers/pull/14075&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@h4iku made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14087"&gt;https://github.com/huggingface/transformers/pull/14087&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@WestonKing-Leatham made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14088"&gt;https://github.com/huggingface/transformers/pull/14088&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@cakiki made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14014"&gt;https://github.com/huggingface/transformers/pull/14014&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@stalkermustang made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14102"&gt;https://github.com/huggingface/transformers/pull/14102&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@iDeepverma made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14091"&gt;https://github.com/huggingface/transformers/pull/14091&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@yeounyi made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14015"&gt;https://github.com/huggingface/transformers/pull/14015&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@jyshdewangan made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14117"&gt;https://github.com/huggingface/transformers/pull/14117&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@karthikrangasai made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/13432"&gt;https://github.com/huggingface/transformers/pull/13432&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@ChainYo made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14059"&gt;https://github.com/huggingface/transformers/pull/14059&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@sergiovalmac made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14155"&gt;https://github.com/huggingface/transformers/pull/14155&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;@huberemanuel made their first contribution in &lt;a href="https://github.com/huggingface/transformers/pull/14142"&gt;https://github.com/huggingface/transformers/pull/14142&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Full Changelog&lt;/strong&gt;: &lt;a href="https://github.com/huggingface/transformers/compare/v4.11.0...v4.12.0"&gt;https://github.com/huggingface/transformers/compare/v4.11.0...v4.12.0&lt;/a&gt;&lt;/p&gt;</subfield>
  </datafield>
  <datafield tag="773" ind1=" " ind2=" ">
    <subfield code="n">url</subfield>
    <subfield code="i">isSupplementTo</subfield>
    <subfield code="a">https://github.com/huggingface/transformers/tree/v4.12.0</subfield>
  </datafield>
  <datafield tag="773" ind1=" " ind2=" ">
    <subfield code="n">doi</subfield>
    <subfield code="i">isVersionOf</subfield>
    <subfield code="a">10.5281/zenodo.3385997</subfield>
  </datafield>
  <datafield tag="024" ind1=" " ind2=" ">
    <subfield code="a">10.5281/zenodo.5608580</subfield>
    <subfield code="2">doi</subfield>
  </datafield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">software</subfield>
  </datafield>
</record>
37,139
1,293
views
downloads
All versions This version
Views 37,139187
Downloads 1,2936
Data volume 10.0 GB73.7 MB
Unique views 30,889151
Unique downloads 6676

Share

Cite as