huggingface/transformers: v4.8.0 Integration with the Hub and Flax/JAX support

v4.8.0 Integration with the Hub and Flax/JAX support Integration with the Hub

Uploading Transformers object to the Hub has never been easier, with all models, tokenizers and configurations having a revamp push_to_hub() method as well as a push_to_hub argument in their save_pretrained() method. The workflow of this method is changed a bit to be more like git, with a local clone of the repo in a folder of the working directory, to make it easier to apply patches (use use_temp_dir=True to clone in temporary folders for the same behavior as the experimental API).

The Trainer also has a push_to_hub API that you need to activate by passing push_to_hub=True in its TrainingArguments. You can control the repo name and organization with push_to_hub_model_id and push_to_hub_organization. This will upload the model, its configuration and the associated tokenizer, the TensorBoard runs (if tensorboard is installed) as well as a draft of model card with the training metrics results on the evaluation set.

• Clean push to hub API #12187 (@sgugger)

Flax/JAX is becoming a fully supported backend of the Transformers library with more models having an implementation in it. BART, CLIP and T5 join the already existing models, find the whole list here.

• [Flax] FlaxAutoModelForSeq2SeqLM #12228 (@patil-suraj)
• [FlaxBart] few small fixes #12247 (@patil-suraj)
• [FlaxClip] fix test from/save pretrained test #12284 (@patil-suraj)
• [Flax] [WIP] allow loading head model with base model weights #12255 (@patil-suraj)
• [Flax] Fix flax test save pretrained #12256 (@patrickvonplaten)
• [Flax] Add jax flax to env command #12251 (@patrickvonplaten)
• add FlaxAutoModelForImageClassification in main init #12298 (@patil-suraj)
• [Flax T5] Fix weight initialization and fix docs #12327 (@patrickvonplaten)

• AutoTokenizer: infer the class from the tokenizer config if possible #12208 (@sgugger)
• update desc for map in all examples #12226 (@bhavitvyamalik)
• Depreciate pythonic Mish and support PyTorch 1.9 version of Mish #12240 (@digantamisra98)
• [t5 doc] make the example work out of the box #12239 (@stas00)
• Better CI feedback #12279 (@LysandreJik)
• Fix for making student ProphetNet for Seq2Seq Distillation #12130 (@vishal-burman)
• [DeepSpeed] don't ignore --adafactor #12257 (@stas00)
• Tensorflow QA example #12252 (@Rocketknight1)
• [tests] reset report_to to none, avoid deprecation warning #12293 (@stas00)
• [trainer + examples] set log level from CLI #12276 (@stas00)
• [tests] multiple improvements #12294 (@stas00)
• Trainer: adjust wandb installation example #12291 (@stefan-it)
• Fix and improve documentation for LEDForConditionalGeneration #12303 (@ionicsolutions)
• [Flax] Main doc for event orga #12305 (@patrickvonplaten)
• [trainer] 2 bug fixes and a rename #12309 (@stas00)
• FlaxBartPretrainedModel -> FlaxBartPreTrainedModel #12313 (@sgugger)
• [docs] performance #12258 (@stas00)
• Add CodeCarbon Integration #12304 (@JetRunner)
• Optimizing away the fill-mask pipeline. #12113 (@Narsil)
• Add output in a dictionary for TF generate method #12139 (@stancld)
• Flax summarization script #12230 (@patil-suraj)
• Rewrite ProphetNet to adapt converting ONNX friendly #11981 (@jiafatom)
• Flax T5 #12150 (@vasudevgupta7)
• Add mention of the huggingface_hub methods for offline mode #12320 (@LysandreJik)
• [Flax/JAX] Add how to propose projects markdown #12311 (@patrickvonplaten)
• [TFWav2Vec2] Fix docs #12283 (@chenht2010)
• Add all XxxPreTrainedModel to the main init #12314 (@sgugger)
• Conda build #12323 (@LysandreJik)
• Changed modeling_fx_utils.py to utils/fx.py for clarity #12326 (@michaelbenayoun)