Run Gemma with LM Studio¶

LM Studio is a desktop app for running local models, with a GUI, a CLI and a local server. It supports Gemma in both GGUF (llama.cpp) and MLX formats.

Install¶

Download for macOS, Windows or Linux from lmstudio.ai/download.

Get a model¶

In the app, press Cmd+Shift+M (macOS) or Ctrl+Shift+M (Windows/Linux), search for "Gemma", and LM Studio suggests the variant that fits your hardware.

From the terminal:

lms get google/gemma-4-12b

List what you have, then load one:

lms ls
lms load <model_key>

lms load takes --gpu=max|auto|0.0-1.0 and --context-length=1-N.

Server¶

lms server start

OpenAI-compatible API at http://localhost:1234/v1, serving /v1/chat/completions, /v1/completions, /v1/embeddings, /v1/models and /v1/responses.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")

response = client.chat.completions.create(
    model="google/gemma-4-12b",
    messages=[{"role": "user", "content": "Say this is a test"}],
)
print(response.choices[0].message.content)

Your own GGUF¶

lms import <path/to/model.gguf>

The model appears under My Models. See Import Models if it isn't detected.