Transformers need to be updated to the latest version.

[Samarth4023]

🚀 Feature

Update Transformers to the latest version to support Llama 3.3 and all upcoming models.

Motivation

The Llama 3.2 model works fine with Transformers version 4.44.2, but when using Llama 3.3, there is an issue loading the tokenizer. The error message generated is:

Exception Traceback (most recent call last)
<ipython-input-43-78a1419bc112> in <cell line: 2>()
      1 # Load tokenizer
----> 2 tokenizer = AutoTokenizer.from_pretrained(model_name)

... 
Exception: data did not match any variant of untagged enum ModelWrapper at line 1251003 column 3

This error does not occur when using Transformers version 4.48.0. This indicates that Llama 3.3 (and potentially other upcoming models) may require an updated version of Transformers to function correctly.

Additional context

• Model: Llama 3.2 works fine with Transformers 4.44.2.
• Model: Llama 3.3 fails with Transformers 4.44.2, but works with 4.48.0.
• The error is related to tokenizer loading, suggesting that higher versions of Transformers are necessary for future models.