Skip to content

Fine-tuning overview

Gemma 4 can reason through problems, call tools, and understand images and audio. Fine-tuning allows you to teach it things it did not know before like domain-specific terminology, a new language or corpus, and capabilities outside of it current skillset. These sets of pages will guide you through fine-tuning, from continued pre-Training and supervised fine-Tuning all the way to reinforcement learning and multi-gpu training.

Fine-tuning is not trivial and might get overwhelming quickly. This page helps you find your way, starting with a map of common paths you can take for fine-tuning Gemma 4 based on specific tasks.

Using a coding agent?

Install the gemma-trainer skill so your agent can prepare your data, pick settings for your GPU, and run fine-tuning for you:

npx skills add google-gemma/gemma-skills --skill gemma-trainer --global

1. The four training stages

LLMs are typically trained in stages. In each stage, the model often sees different kind of data and focuses on different tasks:

A Gemma 4 model passing through four training stages

Here is what happens at each stage:

  1. Pre-training: The model trains on a large corpus of web text, code, images, and audio to predict the next token. The resulting pre-trained checkpoint can continue any document you give it, but it does not follow instructions.
  2. Continued pre-training: You continue training on raw text from your domain, such as medical literature, a new language, or an internal codebase. The model picks up domain knowledge, terminology, and phrasing. See Continued pre-training.
  3. Supervised fine-tuning: You train on example conversations, each pairing a prompt with the exact response you want. See (Q)LoRA and Full fine-tuning.
  4. Reinforcement learning: Instead of imitating a single target answer, the model learns from scored outputs. In preference optimization, it compares chosen and rejected answers. With verifiable rewards, a reward function grades each answer the model generates. See Preference optimization and Verifiable rewards.

You rarely need all four stages. Most teams start from the instruction-tuned (-it) checkpoint, which already follows instructions, and run supervised fine-tuning or reinforcement learning on top.


2. RAG, LoRA, or full fine-tuning

Before you start training, ask yourself whether you actually need to update the weights at all, and if so, whether you need to update all of the model weights.

RAG, LoRA, and full fine-tuning side by side

There are generally three approaches to improving your model:

  • RAG (Retrieval-Augmented Generation): Leaves the model weights untouched and injects retrieved documents directly into the prompt.
  • LoRA (Low-Rank Adaptation): Freezes the base model and trains two small low-rank matrices alongside each layer, updating only about 1% to 2% of the parameters. QLoRA (Quantized Low-Rank Adaptation) uses the exact same adapter technique while storing the frozen base weights in 4-bit precision to save GPU memory.
  • Full fine-tuning: Unfreezes and updates 100% of the model weights during backpropagation.

Use this table to match your goal to the right approach:

Goal Approach Reasoning
Facts that change often, like live inventory or user records RAG Updating a search index takes seconds. Weights should learn skills, not changing facts.
A new language or a large domain corpus Continued pre-training Deep domain shifts benefit from training on raw text before instruction tuning.
A new output format, persona, tool schema, or task skill Supervised fine-tuning Small adapters learn new behavior from a modest chat dataset on a single GPU.
Moving a large model's reasoning into E2B or E4B Supervised fine-tuning Train the compact model on reasoning traces generated by 31B.
Strict correctness, like verifiable code, math, or SQL Reinforcement learning Preference pairs or reward functions catch mistakes that supervised fine-tuning misses.

Combine RAG and LoRA

RAG and fine-tuning complement each other nicely. A pattern that you see quite often is to fine-tune Gemma 4 using LoRA, so it learns your domain and then you can use RAG to give it more recent information.


3. GPU memory

Training requires significantly more GPU memory than inference. For every parameter you train, the GPU must store three values:

  • The weight itself: 2 bytes per parameter in bfloat16.
  • Its gradient: How much the weight should change during the backward pass, another 2 bytes.
  • Optimizer states: AdamW, and optimizer, tracks two running averages per trained weight. 2 bytes with an 8-bit optimizer, or 8 bytes in 32-bit precision.

On top of those come the activations. These are the intermediate values from the forward pass kept in memory and are used to compute gradients during the backward pass.

Together, they give us the following overview when applied to Gemma 4 12B:

GPU memory breakdown for training Gemma 4 12B with full fine-tuning, LoRA, and QLoRA

Each method changes how much memory those components use:

As you can see, different training techniques require different amounts of VRAM. The table below shows peak GPU memory (VRAM) across all five Gemma 4 models at a sequence length of 2,048 tokens:

Model Parameters QLoRA Full fine-tuning
Gemma 4 E2B 2B effective ~7 GB ~32 GB
Gemma 4 E4B 4B effective ~9 GB ~42 GB
Gemma 4 12B 12B ~11 GB ~80 GB
Gemma 4 26B A4B 26B (4B active) ~21 GB ~160 GB
Gemma 4 31B 31B ~22 GB ~200 GB

4. Pre-trained or instruction-tuned

Every Gemma 4 model size is released in two checkpoints:

  • Pre-trained (no suffix), such as google/gemma-4-12B.
  • Instruction-tuned (-it suffix), such as google/gemma-4-12B-it.

A pre-trained model was only trained to continue documents and will generally not respond to instructions. In contrast, an instruction-tuned model will follow instructions, recognizes when to stop, and what the user turns and the model turns are.

The same prompt sent to the pre-trained and the instruction-tuned Gemma 4 12B checkpoints

As such, the google/gemma-4-12B-it is a checkpoint that was fine-tuned from the pre-trained google/gemma-4-12B checkpoint. In most cases, users will start from google/gemma-4-12B-it and fine-tune it further.

Use this rule of thumb when picking a checkpoint:

  • Start from the -it checkpoint for almost everything: Use the -it checkpoint for supervised fine-tuning, tool use, vision and audio tuning, DPO, and GRPO. You build on top of existing instruction-following behavior instead of teaching it from scratch.
  • Start from the pre-trained checkpoint for continued pre-training or custom chat formats: Use a pre-trained checkpoint when you want to train on large amounts of data, add a new domain or language, or when you want to add your own chat template. This requires a significant amount of compute as it generally involves a lot of data to perform various stages of full tuning (continued pre-training, supervised fine-tuning, reinforcement learning, etc.).

5. Training frameworks

Gemma 4 works out of the box with various tools frequently used in the ecosystem:


What's next