Can Local LLMs Complete Multi-Turn Coding Agent Tasks? Benchmarking GGUF, MoE Turn Bloat, and Prompt Caching

A coral-accented robot caught in a tangled loop of gears and hourglasses on the left, contrasted with a calm robot on the right stepping cleanly along a blue path powered by a silver desktop workstation.

Ask a local Ollama model in your terminal to write quicksort or explain a stack trace, and you get an answer in seconds. Open-weight models also score well on single-turn coding benchmarks—in our own 72-task blind single-turn coding suite, qwen3-coder:30b scored 58/72.

Put that same model inside a multi-turn coding agent—where turn one alone ingests roughly 30k tokens of system and tool contracts, and the model has to read files, patch code, and run verification on its own—and the picture changes quickly.

To see what a 64GB unified-memory desktop can actually handle today, we ran controlled multi-turn benchmarks on an Apple M5 Pro (64GB, macOS 26.6.2) using Holon's built-in isolated benchmark suite, comparing local Ollama configurations against a cloud baseline (deepseek-v4-flash).

Up front, several common pieces of local tuning advice did not hold up:


Overall Results and Model Selection

8-Task Benchmark Summary

Model ConfigurationFinal PassUnit Verify (verify)Avg Duration / TaskAvg TurnsAvg Cumulative Input TokensTimeouts / Errors
deepseek-v4-flash (cloud baseline)8/88/821s5.630k0
qwen3.8:27b-gguf (27B Dense, Q4_K_M)6/8–7/88/8422s–437s5.9–6.6218k–248k1–2
gemma4:26b (MoE-A3.8B)5/85/8159s8.5325k0
qwen3-coder:30b (MoE-A3B)3/8–4/87/8561s–583s23.2–36.9962k–1,588k4–5*
qwen3.8:27b-mlx3/84/8552s6.5245k5
qwen3.5:0.8b-mlx0/80/828s8.5327k0

Note: Ranges reflect variance across repeated runs of the same configuration. In qwen3-coder:30b's timeouts, 3 tasks (callback-refactor, fix-multi-file-config-merger, and task-inspection-subagent-status) actually passed underlying unit tests (verify passed), then burned through their budget re-checking files instead of finishing.

What to Pick on a 64GB Desktop

Use CaseRecommended ConfigurationObserved Behavior
Primary coding work (first-try accuracy and clean wrap-up)qwen3.8:27b (ggml-org Q4_K_M GGUF, ~19GB)6/8–7/8 pass, ~6 turns to converge, 5–10 min per task
Everyday quick edits and fast responsesgemma4:26b (MoE, ~3.8B active)5/8 pass with 0 timeouts, 159s/task average, fastest turnaround
Long asynchronous background tasksqwen3-coder:30b (17GB GGUF)Strong code repair capability (verify 7/8), but high turn counts; allow 10–15+ min budgets
Avoid for coding agentsqwen3.8:27b-mlx, qwen3.5:0.8bMLX suffers severe long-prompt prefill latency; 0.8B scored 0/8 and cannot hold tool or patch contracts

Test Environment and Methodology


Inference Backend Comparison: Switching to GGUF Lifts 27B Dense from 3/8 to 7/8

Under the MLX runner, qwen3.8:27b-mlx passed only 3/8 tasks. The remaining 5 failed on time (4 timeouts, 1 failing to finish turn one). Daemon logs showed that on a ~30k-token system prompt, Ollama's MLX runner spent roughly 235 seconds on cold first-turn prefill alone—burning nearly half the task budget before the model read a single file.

To isolate backend execution from quantization differences, we downloaded ggml-org's official Q4_K_M GGUF weights (~19GB, 27B Dense), imported them into Ollama via a local Modelfile using the llama.cpp backend, and re-ran the identical suite and budgets.

All 4 tasks that had timed out under MLX (analyze-pipeline, coupon-constrained-fix, followup-after-multifile-fix, and invoice-pipeline-bug) passed. Total pass rate climbed from 3/8 to 7/8, with verify hitting 8/8. Across repeated runs, aside from an occasional slow opening turn (invoice-pipeline-bug once spiked to 324s on turn one), it held steady at 6/8–7/8.

Telemetry shows the model's reasoning behavior barely changed:

Turn counts and input volumes matched closely. Switching formats did not make the model smarter—llama.cpp/GGUF simply cut first-turn 30k prefill latency enough for capable solutions to finish before the clock ran out.

The only consistent timeout was task-inspection-subagent-status: given a tight 300s budget, two turns took 303s, dominated by first-turn 30k prefill. Short-budget tasks remain highly sensitive to initial prefill delay.


Architecture and Turn Count: Why a 58/72 Single-Turn 30B MoE Stalls at 4/8

qwen3-coder:30b is a 17GB MoE coding model (30.5B total parameters, ~3.3B active per token). It scored 58/72 on our single-turn coding blind test and streams tokens quickly, yet in the isolated multi-turn benchmark it passed only 3/8–4/8 tasks, ballooning to 23.2–36.9 turns and 962k–1,588k input tokens per task.

Logs showed a clear failure mode: fixing the code, then failing to stop. In one 5-timeout run, callback-refactor (72 turns), fix-multi-file-config-merger (41 turns), and task-inspection-subagent-status (7 turns) had already passed unit verification (verify passed). Instead of delivering the result, the model kept issuing read-only shell commands to double-check files until the timeout hit.

The second issue was micro-stepping. qwen3.8:27b-gguf (27B Dense) inspects the relevant files and applies a direct fix in 5.9–6.6 turns. With 3B active parameters, qwen3-coder:30b steps fast, but across multi-file contexts it fragments reads into tiny chunks, probes patches tentatively, and reworks them—multiplying turn count by 4–6×.

This is not an inherent flaw of every MoE model: gemma4:26b (also 3B–4B active) averaged 8.5 turns and 159s per task. The gap comes from qwen3-coder:30b's tendency to hesitate and over-verify inside an agent loop. Because every extra turn replays the accumulated conversation history, fast per-step decoding could not offset 4×–6× more turns.


Prompt Caching and Decode Tuning: What Helps and What Hurts

With first-turn 30k prefill dominating runtimes, we tested daemon keep-alive flags, cross-task prefix caching, and low-level quantization switches on the same hardware.

1. OLLAMA_KEEP_ALIVE=-1 Saves Cold Model Loads, Not First-Turn Prefill

Setting OLLAMA_KEEP_ALIVE=-1 OLLAMA_NUM_PARALLEL=1 OLLAMA_MAX_LOADED_MODELS=1 left average 8-task durations nearly unchanged (qwen3.8:27b-gguf 422s vs. 437s; qwen3-coder:30b 561s vs. 583s). Breaking down 403 prompt eval calls explained why:

2. Cross-Task Prefix Caching Today: Architecture Split and Task-Context Weight

Even when consecutive tasks share an identical system prompt (~83% of the prompt text, over 56k characters), cross-task prefix caching in current llama.cpp (Ollama 0.32.15) hits two practical limits:

Model (Inference Architecture)Cross-Task Opening Cache HitMedian Opening Turn8-Task Opening Sum / Total Provider Time
qwen3-coder:30b (qwen3moe full attention)11,378–14,306 tokens~121s1,237s / 4,296s (29%)
qwen3.8:27b-gguf (qwen35 hybrid linear/full attention)0 (full reprocessing)~156s1,558s / 3,097s (50%)

3. FlashAttention and q8_0 KV Cache Slowed Decode Down

Many local tuning guides recommend setting OLLAMA_FLASH_ATTENTION=1 alongside OLLAMA_KV_CACHE_TYPE=q8_0. Using a controlled 40k prompt + 512-token decode probe, we compared default settings (base) against enabling both flags (fakv):

ModelPrefill Speed (base → fakv)Decode Speed (base → fakv)
qwen3-coder:30b266 → 212 tok/s (-20%)27 → 19.5 tok/s (-28%)
qwen3.8:27b-gguf221.6 → 195.8 tok/s (-12%)11 → 10.2 tok/s (-7%)

The takeaway is straightforward:

  1. On Metal in Ollama 0.32.15, --flash-attn auto is already on by default for supported architectures, so OLLAMA_FLASH_ATTENTION=1 adds no benefit.
  2. The only change came from q8_0 KV cache quantization. While it saves memory, it adds per-token dequantization overhead during generation, cutting qwen3-coder:30b decode throughput by nearly 30%. On a 64GB machine running a 27B–30B Q4_K_M model, memory is not the bottleneck—skip KV quantization.

Local GGUF Import and Configuration Pitfalls

Running ollama run qwen3.8:27b on Apple Silicon defaults to the MLX variant. If you import GGUF weights into Ollama 0.32.15 yourself, watch out for three issues we hit directly:

  1. Do not pipe a Modelfile to ollama create: On Ollama 0.32.15, ollama create <name> -f - reading from stdin fails silently with exit code 0 without creating a usable model. Write an explicit Modelfile to disk (e.g., FROM ./Qwen3.8-27B-Q4_K_M.gguf) and run ollama create qwen3.8:27b-gguf -f ./Modelfile.
  2. Stick to standard ggml-org Q4_K_M GGUF files for manual imports: Some third-party dynamic quantization packs (such as the unsloth quant we tested) failed Ollama's format validation. Standard ggml-org Q4_K_M (~19GB) worked reliably; if you prefer pulling from the Ollama library directly, specify qwen3.8:27b-q4_K_M (the GGUF tag) rather than the bare tag.
  3. Keep Ollama defaults on 64GB Apple Silicon: Leave --flash-attn auto and FP16 KV cache as-is. Skip OLLAMA_KV_CACHE_TYPE=q8_0 (7%–28% slower in our probes), and do not expect OLLAMA_KEEP_ALIVE=-1 to accelerate cross-task first-turn prefill.

Once your local model is ready, follow Run Private Agents in Offline and Self-Hosted Environments: From Ollama to Private Search and Local Toolchains to lock down external network policies, add self-hosted search and local tools, and verify end-to-end private execution. For multi-cloud failover or per-agent routing, see Configure and Switch Models: From Local Ollama to Multi-Cloud Failover, Work in a Project Workspace or Git Worktree, and Manage Agent Memory.


Conclusion

On a 64GB unified-memory desktop paired with llama.cpp/GGUF, a local 27B Dense model (qwen3.8:27b-gguf) already clears 6/8 to 7/8 of realistic multi-turn coding tasks. The remaining bottleneck for local agents is no longer tweaking Ollama environment variables—it comes down to whether models avoid wasted verification loops and wrap up cleanly, and how far we can shrink the 20k+ tokens of unshared per-task context.