From the original Transformer to today’s language and vision models, this is a living map of the releases worth understanding. Start with the milestones, then choose a model to explore in depth.
The selection covers 56 releases and research milestones, checked through October 9, 2026. Selection favors established model families and influential research milestones. It is a reading map, not a complete catalog or a popularity ranking.
To compare ideas across releases, use Architectural Evolution. Its mechanism pages connect memory, position, expert routing, residual paths and visual interfaces back to these studies. Read a deep dive for the model’s full training recipe, implementation choices and evidence.
Major releases at a glance
Bold names mark selected landmarks in architecture, training, or adoption. The 22 highlights are an editorial reading guide, not a ranking.
| Year | Selected models and milestones |
|---|---|
| 2017 | Transformer1 |
| 2018 | GPT2 · BERT3 |
| 2019 | GPT-24 · T55 |
| 2020 | GPT-36 |
| 2021 | CLIP7 · Switch Transformer8 · GLM9 |
| 2022 | InstructGPT10 · Chinchilla11 · PaLM12 · Flamingo13 · BLOOM14 · ChatGPT15 |
| 2023 | BLIP-216 · LLaMA17 · GPT-418 · ChatGLM-6B19 · LLaVA20 · Llama 221 · Mistral 7B22 · Mamba23 · Gemini 1.024 · Mixtral 8x7B25 · InternVL26 |
| 2024 | Gemini 1.5 Pro27 · Claude 328 · DeepSeek-V229 · GPT-4o30 · Llama 3.131 · Qwen2-VL32 · Qwen2.533 · OpenAI o134 · DeepSeek-V335 |
| 2025 | DeepSeek-R135 · Gemma 336 · Gemini 2.5 Pro37 · Qwen338 · Claude 439 · Grok 440 · Kimi K241 · GLM-4.542 · GPT-543 · Qwen3-VL44 |
| 2026 | Qwen3.545 · GLM-546 · Claude Mythos Preview47 · DeepSeek-V435 · Kimi K348 · Qwen3.845 · Claude Fable 5.149 · GPT-6 Astra50 · DeepSeek-V4.1-Flash51 · Claude Opus 5.552 · Gemini 4 Argon53 |
Dates identify the event in the release log: a paper, an announcement, a preview, or a weights release. They do not all mean general availability. Transformer is the architecture anchor; CLIP is a contrastive vision and text model; Mamba is a selective state space architecture, not a Transformer. ChatGPT marks the 2022 research preview.172315
OpenAI o1 records the full model’s December 5, 2024 release in ChatGPT, not the September 12 o1-preview release. Claude Mythos is represented by its April 7, 2026 restricted preview; it is not a general availability event.345447
Model evolution
Read both bands in Figure 1 from left to right. Year placement shows chronology, not lineage. The release log records exact dates and events.
Figure 1. Selected model milestones, 2017 to 2026. AI generated from the cited release list.
Release log: exact dates and event types
| Date | Model | Event recorded |
|---|---|---|
| 2017-06-12 | Transformer1 | Architecture paper |
| 2018-06-11 | GPT2 | Announcement |
| 2018-10-11 | BERT3 | Paper |
| 2019-02-14 | GPT-24 | Staged release |
| 2019-10-23 | T55 | Paper |
| 2020-05-28 | GPT-36 | Paper |
| 2021-01-05 | CLIP7 | Announcement |
| 2021-01-11 | Switch Transformer8 | Paper |
| 2021-03-18 | GLM9 | Paper |
| 2022-01-27 | InstructGPT10 | Alignment announcement |
| 2022-03-29 | Chinchilla11 | Paper |
| 2022-04-05 | PaLM12 | Paper |
| 2022-04-29 | Flamingo13 | Paper |
| 2022-07-12 | BLOOM14 | Weights release |
| 2022-11-30 | ChatGPT15 | Research preview |
| 2023-01-30 | BLIP-216 | Paper |
| 2023-02-27 | LLaMA17 | Paper |
| 2023-03-14 | GPT-418 | Announcement |
| 2023-03-14 | ChatGLM-6B19 | Weights release |
| 2023-04-17 | LLaVA20 | Paper |
| 2023-07-18 | Llama 221 | Family announcement |
| 2023-09-27 | Mistral 7B22 | Weights release |
| 2023-12-01 | Mamba23 | Paper |
| 2023-12-06 | Gemini 1.024 | Family announcement |
| 2023-12-11 | Mixtral 8x7B25 | Release article |
| 2023-12-21 | InternVL26 | Paper |
| 2024-02-15 | Gemini 1.5 Pro27 | Limited preview |
| 2024-03-04 | Claude 328 | Family announcement |
| 2024-05-07 | DeepSeek-V229 | Paper |
| 2024-05-13 | GPT-4o30 | Announcement |
| 2024-07-23 | Llama 3.131 | Weights release |
| 2024-08-29 | Qwen2-VL32 | Announcement |
| 2024-09-19 | Qwen2.533 | Family announcement |
| 2024-12-05 | OpenAI o134 | ChatGPT release |
| 2024-12-26 | DeepSeek-V335 | API release |
| 2025-01-20 | DeepSeek-R135 | API release |
| 2025-03-12 | Gemma 336 | Family announcement |
| 2025-03-25 | Gemini 2.5 Pro37 | Experimental release |
| 2025-04-29 | Qwen338 | Family announcement |
| 2025-05-22 | Claude 439 | Family announcement |
| 2025-07-09 | Grok 440 | Announcement |
| 2025-07-11 | Kimi K241 | Weights release |
| 2025-07-28 | GLM-4.542 | Announcement |
| 2025-08-07 | GPT-543 | Announcement |
| 2025-09-23 | Qwen3-VL44 | First family weights |
| 2026-02-16 | Qwen3.545 | First family weights |
| 2026-02-17 | GLM-546 | Paper |
| 2026-04-07 | Claude Mythos Preview47 | Restricted preview |
| 2026-04-24 | DeepSeek-V435 | API preview |
| 2026-07-27 | Kimi K348 | Weights and report |
| 2026-08-12 | Qwen3.845 | First family weights |
| 2026-09-01 | Claude Fable 5.149 | Model release |
| 2026-09-03 | GPT-6 Astra50 | Initial rollout |
| 2026-09-10 | DeepSeek-V4.1-Flash51 | API release |
| 2026-09-22 | Claude Opus 5.552 | Announcement |
| 2026-09-30 | Gemini 4 Argon53 | Restricted rollout |
Some families arrived in stages. For example, GPT-2 initially released smaller models, Gemini 2.5 Pro began as an experimental release, and Gemini 4 Argon initially rolled out to trusted cyber defenders.43753 Qwen2-VL uses the announcement’s August 29 date; its repository records August 30 for the initial smaller model release.3244
Choose a deep dive
Studies of models with public weights follow the Model Deep Dive guide. It connects technical reports to named artifacts and distinguishes reported findings from inspected implementation and measured results.
The first collection contains 41 studies, selected for distinct technical lessons rather than one page per checkpoint. Each starts with the original report and Hugging Face links, then highlights and scale. Architecture, training, inference and evaluation connect to the inspected sources. Original LLaMA is a historical study with restricted release terms; public weights elsewhere also have different licenses and access rules. Each page states its own verification boundary. These are source studies, not locally reproduced model benchmarks.
Browse related generations under GPT, Llama, Mistral, DeepSeek, Qwen, GLM, Kimi, Gemma and MiniMax in the sidebar. Single studies stay directly accessible. The reading paths below connect models by the ideas they teach, across family boundaries.
Language foundations
| Study | What to understand |
|---|---|
| GPT-2 | Causal prediction, residual paths and generation state |
| BERT | Bidirectional representations and masked prediction |
| T5 | One text-to-text interface, encoder-decoder computation |
| LLaMA | A practical dense-decoder recipe and its disclosure limits |
| Llama 3 / 3.1 | Scaling the recipe, data and long-context extension |
| Mistral 7B | Sliding-window attention and grouped-query memory |
| GLM / GLM-130B | Autoregressive blank filling and stable bilingual scaling |
| GLM-4 | Distinguishing a released 9B model from a wider API family |
| Qwen2.5 | Data scaling, staged alignment and explicit generation scope |
| Gemma | Compact dense models and a practical open weight recipe |
| Gemma 2 | Distillation, attention patterns and training efficiency |
Experts and reasoning
| Study | What to understand |
|---|---|
| Mixtral 8x7B | Sparse expert routing and total versus active size |
| DeepSeek-V2 | Latent attention and fine-grained experts |
| DeepSeek-V3 | Load balancing, FP8 training and multi-token prediction |
| DeepSeek-R1 | Reasoning RL, data curation and distillation |
| Qwen3 | Thinking and direct-answer modes in one generation |
| GLM-5 | Sparse history selection and asynchronous agent learning |
| Kimi K2 | Large-scale experts, Muon and agent data |
| Kimi K3 | Mixing across sequence, depth and expert width |
| DeepSeek-V4 | Sequence compression and constrained residual mixing |
| DeepSeek-V4.1-Flash | Causal encoding, shared caches and approximate replay |
| gpt-oss | Quantized experts, mixed attention and the Harmony agent interface |
| GLM-4.5 | Expert capacity, agent tasks and staged reasoning alignment |
| MiniMax M1 | CISPO and numerical consistency in long reasoning RL |
| MiniMax M2 Series | Full attention, verifiable environments and asynchronous agent RL |
Memory architectures
| Study | What to understand |
|---|---|
| Mamba | Input-dependent state updates instead of an attention cache |
| Kimi Linear | Corrective recurrent memory alongside global lookup |
| Qwen2.5 1M | Long context training and sparse serving versus dense checkpoints |
| Qwen3.8 Flash Next | Sparse experts and recurrent attention in the released backbone |
| MiniMax 01 | Lightning attention, exact history and the long context curriculum |
Vision and language
| Study | What to understand |
|---|---|
| CLIP | Contrastive alignment, not visual text generation |
| BLIP-2 | Learned queries connecting frozen encoders and language models |
| LLaVA / 1.5 | Visual instruction tuning and a simpler connector |
| Qwen2-VL | Dynamic resolution and multimodal position |
| Qwen3-VL | Deep visual injection, position and staged training |
| Gemma 3 | Compact image views and local/global attention |
| InternVL 3 | Native multimodal pretraining and a changing visual budget |
| Qwen2.5-VL | Resolution, video time and document grounding |
| GLM-4V | Shared vision training ideas across dense and expert backbones |
| Gemma 4 | Native multimodal interfaces, model variants and prediction modules |
| MiniMax M3 / MSA | Learned block selection and the native VLM release boundary |
Related releases and study boundaries
This is a curated technical collection, not a page for every checkpoint update. Qwen3 Next, Qwen3.5 and Gemma 3n have useful official technical documentation, but this expansion did not verify a matching standalone generation report. See Qwen3 Next, Qwen3.5 and Gemma 3n for those documentation boundaries. A backbone paper does not establish a later model’s entire vision or alignment recipe.
Pick a reading path
- Understand the backbone: GPT-2, BERT, then T5.
- Follow efficient memory: Mistral, DeepSeek-V2, Mamba, then V4.1-Flash.
- Connect vision to language: CLIP, BLIP-2, LLaVA, then Qwen3-VL.
- Study reasoning and agents: V3, R1, Qwen3, then GLM-5.
Use Transformer Foundations for the shared mechanisms and Architectural Evolution to compare design choices. This overview will grow as new releases add useful contrasts.
How we read a model
Follow the input from its processor through the model to the output. Compare the report with the released configuration and source code, then examine the training disclosure and evaluation conditions. Keep missing details visible: a model name does not establish a checkpoint, a complete recipe, or a universal performance ranking.
References
-
Vaswani et al. Attention Is All You Need. 2017. ↩ ↩2 ↩3
-
OpenAI. Improving language understanding with unsupervised learning. 2018. ↩ ↩2
-
Devlin et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. 2018. ↩ ↩2
-
OpenAI. Better language models and their implications. 2019. ↩ ↩2 ↩3
-
Raffel et al. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. 2019. ↩ ↩2
-
Brown et al. Language Models are Few-Shot Learners. 2020. ↩ ↩2
-
OpenAI. CLIP: Connecting text and images. 2021. ↩ ↩2 ↩3
-
Fedus et al. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. 2021. ↩ ↩2
-
Du et al. GLM: General Language Model Pretraining with Autoregressive Blank Infilling. 2021. ↩ ↩2
-
OpenAI. Aligning language models to follow instructions. 2022. ↩ ↩2
-
Hoffmann et al. Training Compute-Optimal Large Language Models. 2022. ↩ ↩2
-
Chowdhery et al. PaLM: Scaling Language Modeling with Pathways. 2022. ↩ ↩2
-
Alayrac et al. Flamingo: a Visual Language Model for Few-Shot Learning. 2022. ↩ ↩2
-
BigScience / Hugging Face. Introducing The World’s Largest Open Multilingual Language Model: BLOOM. 2022. ↩ ↩2
-
OpenAI. Introducing ChatGPT. 2022. ↩ ↩2 ↩3
-
Li et al. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. 2023. ↩ ↩2
-
Touvron et al. LLaMA: Open and Efficient Foundation Language Models. 2023. ↩ ↩2
-
Team GLM. ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. 2024, Section 1 and Table 1 record the 2023 release. ↩ ↩2
-
Liu et al. Visual Instruction Tuning. 2023. ↩ ↩2
-
Meta. Meta and Microsoft Introduce the Next Generation of Llama. 2023. ↩ ↩2
-
Mistral AI. Mistral 7B. 2023. ↩ ↩2
-
Gu and Dao. Mamba: Linear-Time Sequence Modeling with Selective State Spaces. 2023. ↩ ↩2 ↩3
-
Google / Google DeepMind. Introducing Gemini: our largest and most capable AI model. 2023. ↩ ↩2
-
Mistral AI. Mixtral of experts. 2023. ↩ ↩2
-
Chen et al. InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks. 2023. ↩ ↩2
-
Google / Google DeepMind. Our next-generation model: Gemini 1.5. 2024. ↩ ↩2
-
Anthropic. Introducing the next generation of Claude. 2024. ↩ ↩2
-
DeepSeek-AI. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model. 2024. ↩ ↩2
-
OpenAI. Hello GPT-4o. 2024. ↩ ↩2
-
Meta. Introducing Llama 3.1: Our most capable models to date. 2024. ↩ ↩2
-
Qwen Team. Qwen2-VL: To See the World More Clearly. 2024. ↩ ↩2 ↩3
-
Qwen Team. Qwen2.5: A Party of Foundation Models!. 2024. ↩ ↩2
-
OpenAI. Introducing ChatGPT Pro. 2024. ↩ ↩2 ↩3
-
DeepSeek. Change Log. Dated release notes, inspected October 9, 2026. ↩ ↩2 ↩3 ↩4 ↩5 ↩6
-
Google DeepMind. Introducing Gemma 3: The most capable model you can run on a single GPU or TPU. 2025. ↩ ↩2
-
Google DeepMind. Gemini 2.5: Our most intelligent AI model. 2025. ↩ ↩2 ↩3
-
Qwen Team. Qwen3: Think Deeper, Act Faster. 2025. ↩ ↩2
-
Anthropic. Introducing Claude 4. 2025. ↩ ↩2
-
Moonshot AI. Kimi K2: Open Agentic Intelligence and dated research index. 2025. ↩ ↩2
-
Z.ai. Z.ai Releases GLM-4.5. 2025. ↩ ↩2
-
OpenAI. Introducing GPT-5. 2025. ↩ ↩2
-
Qwen Team. Qwen3-VL repository. Dated release notes, inspected October 9, 2026. ↩ ↩2 ↩3
-
Qwen Team. Qwen3.5, Qwen3.6 and Qwen3.8 release notes. Dated release notes, inspected October 9, 2026. ↩ ↩2 ↩3 ↩4
-
GLM-5 Team. GLM-5: from Vibe Coding to Agentic Engineering. 2026. ↩ ↩2
-
Anthropic. Project Glasswing. 2026, April 7 launch and early access description. ↩ ↩2 ↩3
-
Moonshot AI. Kimi K3 Open Day: weights, technical report and infrastructure. 2026. ↩ ↩2
-
Anthropic. Claude Fable 5.1 and Claude Fable and Mythos 5.1. 2026. ↩ ↩2
-
OpenAI. Safety overview: GPT-6 Astra. 2026. ↩ ↩2
-
DeepSeek. Introducing DeepSeek V4.1 Flash: smarter, faster, more efficient. 2026. ↩ ↩2
-
Anthropic. Introducing Claude Opus 5.5. 2026. ↩ ↩2
-
Google DeepMind. Gemini 4 Argon: our next era of frontier intelligence. 2026. ↩ ↩2 ↩3
-
OpenAI. Introducing OpenAI o1-preview. 2024. ↩

