Learn¶
This section explains how the Gemma models actually work. Where the rest of the documentation shows you what to run, these pages take an architecture apart and draw it — what the layers do, why a design choice was made, and where a model's behaviour comes from.
Read them in any order. Nothing here is a prerequisite for using Gemma.
What's next¶
How the architecture works, drawn out step by step.
What the unified 12B model does differently.
Drafter models and speculative decoding, illustrated.
How denoising produces text instead of pixels.
Previous Gemma Explained Series¶
Text-to-text models and specialization for coding.
Updated architecture with knowledge distillation for 2B and 9B models.
The Griffin architecture mixing local attention and linear recurrences for fast long-sequence inference.
Vision-language model that takes text and images as input to generate text outputs.
Updated vision-language model with 5-to-1 interleaved attention and 128k context handling.
Text embedding model for search and retrieval.