Ollama

Run open models locally, offline.

Ollama is taught here in 12 lessons, running from Running models locally through to Troubleshooting and the limits of local models. Each lesson takes one topic, shows the working code, and links onward to the next, so the course can be read straight through in order.

Lessons

1. Running models locallyInstall Ollama, pull a model, and understand why local inference is a different trade-off rather than a free replacement.
2. Calling Ollama from codeThe HTTP API, streaming responses, embeddings, and a small wrapper you can reuse across a project.
3. Choosing and evaluating modelsHow to pick a local model by task rather than by leaderboard, and how to test it against your own data.
4. The Modelfile and custom modelsWrite a Modelfile with FROM, SYSTEM, TEMPLATE and PARAMETER, set a reusable system prompt, and build a custom model with ollama create.
5. Model tags, sizes and quantisation explainedThe model:tag convention, parameter counts, Q4/Q5/Q8 and FP16 variants, VRAM versus RAM, and pinning tags so a deployment cannot drift.
6. Managing models from the CLI and APIpull, list, ps, show, copy, rm and push, inspecting details and templates, and scripting model provisioning for a fleet of machines.
7. Local RAG with Ollama embeddingsEmbedding models such as nomic-embed-text, batched /api/embed calls, storing vectors, sensible chunking, and answering from your own documents offline.
8. Structured output and tool callingThe format parameter for JSON and JSON schemas, grammar-constrained decoding, tools in /api/chat, and validating what comes back.
9. Performance tuningkeep_alive, concurrency and model-load limits, context and output caps, GPU offload, and measuring tokens per second instead of guessing.
10. The OpenAI-compatible endpointPoint an OpenAI SDK at /v1, the chat completions and embeddings shapes, which features work, and which silently do not.
11. Serving Ollama in Docker and over a networkContainer images, GPU passthrough, volumes for models, OLLAMA_HOST binding, reverse proxies, and access control that is not optional.
12. Troubleshooting and the limits of local modelsModel not found, out-of-memory and slow loads, context overflow, context rot in small models, and the honest point at which a hosted API is the right answer.

More in AI & Intelligent Development

AI Basics AI Agents Math for AI Machine Learning scikit-learn TensorFlow PyTorch Natural Language Processing LangChain OpenCV Codex Claude Code OpenCode Vibe Coding Selenium Playwright

FAQ