Run Gemma with Ollama¶

Ollama downloads quantized weights for you and exposes a CLI plus a local HTTP API. No GPU required, though one helps.

Install¶

Download from ollama.com/download, then check it worked:

ollama --version

Run¶

ollama run gemma4:12b "Explain quantum computing simply"

Downloads the model on first use. To fetch without running:

ollama pull gemma4:12b

Model tags¶

gemma4:latest is an alias for e4b. On Apple Silicon, the -mlx tags (gemma4:12b-mlx) use the MLX runtime. Full list: ollama.com/library/gemma4/tags.

Images¶

ollama run gemma4:12b ./image.png "What is in this image?"

Server¶

Ollama serves on port 11434.

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1/",
    api_key="ollama",  # required but ignored
)

response = client.chat.completions.create(
    model="gemma4:12b",
    messages=[{"role": "user", "content": "Say this is a test"}],
)
print(response.choices[0].message.content)

curl http://localhost:11434/api/generate -d '{
  "model": "gemma4:12b",
  "prompt": "roses are red"
}'

IMG=$(base64 < test.jpg | tr -d '\n')

curl -X POST http://localhost:11434/api/chat \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4:12b",
    "messages": [{
      "role": "user",
      "content": "What is in this image?",
      "images": ["'"$IMG"'"]
    }],
    "stream": false
  }'