Running local models with Ollama and LM Studio

Point OpenCode at a local endpoint, understand why small models break tool calling, and when local inference is genuinely the right trade.

Wiring a local endpoint

Both Ollama and LM Studio expose an OpenAI-compatible HTTP endpoint. OpenCode reaches them through the OpenAI-compatible adapter, so the configuration is a base URL plus the model names you want to expose.

# Ollama: pull something and start the server
ollama pull qwen2.5-coder:7b
ollama serve            # listens on http://localhost:11434

# LM Studio: load a model in the GUI, start the local server
# it listens on http://127.0.0.1:1234 by default

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": { "baseURL": "http://localhost:11434/v1" },
      "models": {
        "qwen2.5-coder:7b": { "name": "Qwen 2.5 Coder 7B" },
        "llama3.2": { "name": "Llama 3.2" }
      }
    },
    "lmstudio": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "LM Studio (local)",
      "options": { "baseURL": "http://127.0.0.1:1234/v1" },
      "models": {
        "qwen2.5-coder-14b-instruct": { "name": "Qwen 2.5 Coder 14B" }
      }
    }
  }
}

The keys under models must match the names the local server reports, including tags such as :7b.
Local providers need no API key, but some gateways still require a placeholder value - a dummy string is enough.
ollama ps shows whether the model is resident. An unloaded model adds a long first-token delay to the first tool call.

Tool calling is where small models fail

An agent is not a chat model: it must emit correctly formed tool calls, read the results, and decide what to do next. Many models that chat fluently fall apart here, producing malformed arguments, inventing tool names, or answering in prose when a tool call was required.

Rough size	Tool calling	Realistic use
3B and below	Usually unreliable	Commit messages, summaries, no tools
7B coder-tuned	Workable for simple tools	Single-file edits with a narrow prompt
14B	Reliable enough for most loops	Multi-file work in a familiar codebase
30B and above	Comparable to small hosted models	Full agent workflows, if you have the hardware

💡

Test the loop, not the chat. Ask a local model to read one file, state what it changed, and make a single-line edit. A model that cannot complete that reliably will not become trustworthy through better prompting - pick a larger one.

Cost, privacy and speed

# keep the model resident so the first call is not a load
OLLAMA_KEEP_ALIVE=30m ollama serve

# a quick throughput check before you commit a workflow to it
curl -s http://localhost:11434/api/generate -d '{
  "model": "qwen2.5-coder:7b",
  "prompt": "print hello",
  "stream": false
}' | head -c 200

Private by construction: no prompt, file or diff leaves the machine.
Free per token, but you pay in wall-clock time - local generation is often several times slower than a hosted model.
Good fit for confidential repositories, bulk mechanical edits and offline work; a poor fit for hard reasoning or anything latency-sensitive.
You can mix: local for the build agent, hosted for the plan agent, in the same configuration file.

Situation	Local	Hosted
Source that must not leave the network	Yes	No
Latency-sensitive interactive work	Usually no	Yes
Large mechanical refactor, budget-capped	Yes	Possible
Architecture planning	Rarely enough capability	Yes

FAQ

Why does OpenCode complain that my local model has no tools?

The model is not advertising tool-calling support, or it is emitting malformed calls. Confirm the model is a coder-tuned instruction model of adequate size, and check that the local server really is the OpenAI-compatible endpoint at the /v1 path rather than a chat-only route.

Is a local model good enough to replace a hosted one?

For narrow, well-specified edits on a familiar codebase, a 14B class model can be adequate. For open-ended feature work and debugging across an unfamiliar repository it is not, and the extra attempts cost more time than the API calls would have.

Choosing and configuring model providers Plan mode versus build mode

Last refreshed 2026-09-18.

Running local models with Ollama and LM Studio

Wiring a local endpoint

Tool calling is where small models fail

Cost, privacy and speed

FAQ

Related