Claude Code and Ollama: Local LLMs Alongside Claude

Claude Code is the right tool for hard reasoning, multi-file refactors, and anything where you need a model that actually understands your codebase. But you do not need to burn Claude tokens on every single task. For embedding generation, simple classification, fast autocomplete, or air-gapped environments, a local model running through Ollama is usually the better call.

This post covers how to wire Claude Code and Ollama together so each one does what it is best at. We will look at when to delegate, how to share context between them, and the failure modes you should plan for.

Why pair Claude Code with Ollama at all

Claude Code lives in your terminal and operates at the level of "ship this feature" or "refactor the auth layer". It reads files, runs commands, and reasons across your project. Ollama is a runtime for local open-weight models like Llama 3, Qwen, DeepSeek, Mistral, and Gemma. It exposes a simple HTTP API on localhost:11434 and runs entirely on your machine.

Pairing them is not about replacing Claude. It is about routing work to the cheapest competent model:

• Claude Code handles planning, architecture, multi-file edits, debugging, anything that needs reasoning.
• Ollama handles bulk inference: embeddings, log classification, doc summarisation, local autocomplete, offline fallback.

The economics matter at scale. If you generate 50,000 embeddings a day for a vector search index, you do not want any of that traffic hitting a frontier model. A local nomic-embed-text model on Ollama does the job for free.

Installing Ollama next to Claude Code

Ollama is a single binary. On macOS or Linux:

curl -fsSL https://ollama.com/install.sh | sh
ollama serve &
ollama pull llama3.1:8b
ollama pull nomic-embed-text

That gives you a chat-capable model and an embedding model. You can verify with:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Why is the sky blue?",
  "stream": false
}'

If that returns JSON with a response field, you are ready to wire it into Claude Code workflows. For more on local-first setups, see our guide to Claude Code on Linux and the Mac setup walkthrough.

Pattern 1: Ollama as a delegated tool

The cleanest integration is to expose Ollama to Claude Code as a callable tool. You write a thin shell wrapper that Claude can invoke when it decides a task is below the threshold of "needs Claude".

Create ~/bin/ask-local:

#!/usr/bin/env bash
# ask-local: route a quick question to a local Ollama model
MODEL="${OLLAMA_MODEL:-llama3.1:8b}"
PROMPT="$1"

curl -sS http://localhost:11434/api/generate \
  -d "{\"model\": \"$MODEL\", \"prompt\": $(jq -Rs . <<< "$PROMPT"), \"stream\": false}" \
  | jq -r '.response'

chmod +x ~/bin/ask-local. Then in your CLAUDE.md:

## Local Inference

For trivial completions, regex generation, simple text classification, or
log-line summarisation, prefer `ask-local "..."` over reasoning through
the answer yourself. Reserve your own attention for hard problems.

Claude Code will now read that instruction and shell out to Ollama when the task is small enough. You can also configure this through custom subagents for more structured delegation.

Pattern 2: Embeddings pipeline

The single most common reason to add Ollama to a Claude Code project is local embeddings. You want a vector index for code search, document RAG, or semantic deduplication, but you do not want to pay per-token for embeddings forever.

import requests, sqlite3, json
import numpy as np

def embed(text: str) -> list[float]:
    r = requests.post(
        "http://localhost:11434/api/embeddings",
        json={"model": "nomic-embed-text", "prompt": text},
        timeout=30,
    )
    r.raise_for_status()
    return r.json()["embedding"]

# Pair with sqlite-vec or any vector store
conn = sqlite3.connect("vectors.db")
conn.enable_load_extension(True)
conn.load_extension("vec0")

# Claude Code writes the rest of the pipeline:
# - chunk source files
# - call embed() per chunk
# - upsert into vec0 table
# - expose search() to your CLI

Ask Claude Code to build the chunking, ingestion, and search layer around the embed() call. It handles the schema, batching, and error retries. Ollama handles the bulk inference. Neither model is doing the other's job.

If you want a managed alternative for production, our post on Claude Code with Neon covers Postgres + pgvector setups.

Get Claudify. The official Claude Code starter kit, configured for hybrid local and cloud LLM workflows.

Pattern 3: Offline fallback for CI

Claude Code is great in local development, but you do not want your CI pipeline to depend on the Anthropic API for tasks like "generate a release note from this diff" or "classify this changelog entry". If the API has a regional outage, your build should not break.

The pattern is a two-tier fallback in a script:

#!/usr/bin/env bash
# generate-summary.sh
DIFF=$(git diff HEAD~1 HEAD)

# Try Claude first if a key is set
if [ -n "$ANTHROPIC_API_KEY" ]; then
  SUMMARY=$(echo "$DIFF" | claude-summarise 2>/dev/null) || SUMMARY=""
fi

# Fall back to Ollama if Claude failed or no key
if [ -z "$SUMMARY" ]; then
  SUMMARY=$(echo "$DIFF" | ask-local "Summarise this diff in two sentences:")
fi

echo "$SUMMARY"

This is not about quality parity. The Ollama output will be worse than Claude's. But "worse summary" beats "broken pipeline" every time. The fallback exists so the build keeps running.

Pattern 4: Mixing models in a Claude Code agent

You can also build custom Claude Code subagents that route to Ollama internally. The subagent definition lives in .claude/agents/log-classifier.md:

---
name: log-classifier
description: Classifies log lines into severity buckets. Uses local model.
tools: Bash
---

You classify log lines. For each line, output one of: INFO, WARN, ERROR, FATAL.

For each line, run:
  ask-local "Classify this log line as INFO, WARN, ERROR, or FATAL: <LINE>"

Return the classifications as a JSON array.

Claude orchestrates. Ollama answers. The subagent abstracts the boundary so the rest of your codebase does not need to know which model did what.

Choosing the right local model

Ollama supports dozens of model families. A few defaults that work well alongside Claude Code:

Task	Model	Size	Notes
General chat	llama3.1:8b	4.7GB	Strong default, fast on M-series Macs
Code completion	qwen2.5-coder:7b	4.7GB	Tuned for code, supports FIM
Embeddings	nomic-embed-text	274MB	768-dim, fast, good for code search
Classification	phi3.5:mini	2.2GB	Tiny, surprisingly capable for short tasks
Long context	qwen2.5:14b	9GB	128k context, slower but solid

Pull what you need with ollama pull <model>. Disk is the only cost. You can switch the default model per script by setting OLLAMA_MODEL before calling ask-local.

Failure modes to plan for

Three things will go wrong:

Ollama is not running. Wrap your calls. A curl --connect-timeout 2 and a clean error message beats a 30-second hang. The simplest guard:

if ! curl -sS --connect-timeout 1 http://localhost:11434/api/tags > /dev/null; then
  echo "Ollama not running. Start with: ollama serve" >&2
  exit 1
fi

The local model hallucinates worse than Claude. This is real. Llama 3.1 8b will confidently make up function names or invert booleans. For anything that touches production code, keep Claude in the loop. Use Ollama for input data that you can validate (classification labels, embeddings) not output that gets committed.

Disk and RAM. A 7B model uses ~5GB of RAM during inference and around 5GB on disk. If you pull half a dozen models, you will fill a laptop SSD. ollama list shows what you have. ollama rm <model> reclaims space.

When not to use Ollama

A few cases where you should stick with Claude Code alone:

• Reasoning over your codebase. Claude has the project context. A local model does not.
• Multi-step debugging. This needs frontier-level reasoning. Llama will go in circles.
• Anything safety-critical. Auth, payment flows, data migrations. Use the strongest model available.
• Low-volume tasks. If you are running 10 inferences a day, the token cost is negligible. The overhead of maintaining a local pipeline is not worth it.

Our broader take on model selection is in Claude Code model comparison.

A pragmatic hybrid setup

Most teams that integrate Ollama end up with a setup like this:

1. Claude Code as the primary interface in the terminal.
2. Ollama running as a background service, auto-started at login.
3. A small set of shell wrappers (ask-local, embed-local, classify-local) that the team agrees on.
4. CLAUDE.md instructing Claude when to delegate.
5. A monitoring line in ~/.zshrc or ~/.bashrc that checks Ollama is healthy on shell startup.

That is the whole pattern. Nothing fancy. Claude does the thinking, Ollama does the bulk grunt work, and you stop wasting frontier tokens on tasks that do not need them.

Conclusion

Claude Code and Ollama are not competitors. They are different layers of the same stack. Use Ollama for embeddings, classification, and offline fallback. Use Claude Code for everything that actually needs to think. The split saves money, reduces latency for trivial calls, and gives you a working system even when the network is down.

If you want a curated set of Claude Code skills and a working CLAUDE.md that already understands hybrid model routing, that is exactly what Claudify ships.

Get Claudify. The official Claude Code starter kit for developers who want hybrid local and cloud workflows out of the box.