The Modelfile and custom models

Write a Modelfile with FROM, SYSTEM, TEMPLATE and PARAMETER, set a reusable system prompt, and build a custom model with ollama create.

Writing a Modelfile

# Modelfile — a support assistant with a fixed system prompt
FROM llama3.2:3b-instruct-q4_K_M

SYSTEM """
You are a support assistant for an online store.
Answer only from the policy text provided by the user.
If the policy does not contain the answer, reply exactly: NOT_IN_POLICY.
Keep answers under 60 words. Use British English.
"""

PARAMETER temperature 0.1
PARAMETER top_p 0.9
PARAMETER num_ctx 4096

# stop sequences end generation early and cheaply
PARAMETER stop "<|eot_id|>"
PARAMETER stop "User:"

# the chat template for an instruction-tuned model
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
{{ .Response }}<|eot_id|>"""

LICENSE """
Internal use only. Base model licence applies.
"""

# validate before building
ollama create support-bot -f ./Modelfile
ollama run support-bot
ollama show support-bot --modelfile      # print the resolved Modelfile
ollama show support-bot --parameters     # the effective parameter values

• FROM accepts a registry model, a local model name, or a path to a GGUF file. Pinning a tag makes the build reproducible; FROM llama3.2 will silently change under you.
• SYSTEM is prepended to every conversation, so a long system prompt costs tokens on every request. Keep it tight.
• PARAMETER values are defaults, not limits: a client can override temperature per request through the API.
• TEMPLATE must match what the base model was trained with. Getting the special tokens wrong produces a model that answers in the wrong format rather than an error, which makes it hard to notice in a short test.

Building without the CLI

import requests

MODELFILE = """FROM llama3.2:3b-instruct-q4_K_M
SYSTEM """You are a terse technical editor. Fix grammar only."""
PARAMETER temperature 0.0
PARAMETER num_ctx 2048
"""

def create_model(name, modelfile, base="http://localhost:11434"):
    response = requests.post(
        f"{base}/api/create",
        json={"model": name, "modelfile": modelfile, "stream": False},
        timeout=600,
    )
    response.raise_for_status()
    return response.json()

print(create_model("copy-editor", MODELFILE))

# confirm what was actually built
details = requests.post("http://localhost:11434/api/show",
                        json={"model": "copy-editor"}).json()
print(details["parameters"])
print(details["template"][:120])

• Store the Modelfile in version control next to the application. A model that exists only on someone's laptop is not a deployable artefact.
• Rebuilding from a different base tag produces a different model with the same name. Record the base digest, which ollama show reports.
• Custom models are stored locally and can be pushed to the Ollama registry with ollama push if you have an account and namespace.
• /api/create with stream: true (the default) emits progress lines. Set it to false in scripts, or parse the newline-delimited JSON.

Swapping prompts without rebuilding

# a Modelfile is convenient, but most prompt iteration should happen in code
import requests

def ask(prompt, system=None, model="llama3.2", **options):
    payload = {"model": model, "prompt": prompt, "stream": False, "options": options}
    if system:
        payload["system"] = system            # overrides the Modelfile SYSTEM
    return requests.post("http://localhost:11434/api/generate",
                         json=payload, timeout=120).json()["response"]

policy = "Refunds: 30 days from delivery. Gift cards are non-refundable."

print(ask(
    f"Policy:\n{policy}\n\nQuestion: Can I refund a gift card?",
    system="Answer only from the policy. Reply NOT_IN_POLICY if absent.",
    temperature=0.0,
))

# keep the Modelfile for things that never change: base model, template, context size.
# Keep the system prompt in code when it changes weekly.

FAQ

Related

Running models locally Model tags, sizes and quantisation explained

Last refreshed 2026-09-18.