diff --git a/chapters/en/chapter1/4.mdx b/chapters/en/chapter1/4.mdx index 3870b541f..a0394b327 100644 --- a/chapters/en/chapter1/4.mdx +++ b/chapters/en/chapter1/4.mdx @@ -34,8 +34,7 @@ The [Transformer architecture](https://arxiv.org/abs/1706.03762) was introduced - **May 2020**, [GPT-3](https://huggingface.co/papers/2005.14165), an even bigger version of GPT-2 that is able to perform well on a variety of tasks without the need for fine-tuning (called _zero-shot learning_) -- **January 2022**: [InstructGPT](https://huggingface.co/papers/2203.02155), a version of GPT-3 that was trained to follow instructions better -This list is far from comprehensive, and is just meant to highlight a few of the different kinds of Transformer models. Broadly, they can be grouped into three categories: +- **January 2022**: [InstructGPT](https://huggingface.co/papers/2203.02155), a version of GPT-3 that was trained to follow instructions better. - **January 2023**: [Llama](https://huggingface.co/papers/2302.13971), a large language model that is able to generate text in a variety of languages. @@ -45,6 +44,9 @@ This list is far from comprehensive, and is just meant to highlight a few of the - **November 2024**: [SmolLM2](https://huggingface.co/papers/2502.02737), a state-of-the-art small language model (135 million to 1.7 billion parameters) that achieves impressive performance despite its compact size, and unlocking new possibilities for mobile and edge devices. + +This list is far from comprehensive, and is just meant to highlight a few of the different kinds of Transformer models. Broadly, they can be grouped into three categories: + - GPT-like (also called _auto-regressive_ Transformer models) - BERT-like (also called _auto-encoding_ Transformer models) - T5-like (also called _sequence-to-sequence_ Transformer models)