On this page
Run Private Agents in Offline and Self-Hosted Environments: From Ollama to Private Search and Local Toolchains
This guide shows how to run Holon without public cloud LLMs or external SaaS dependencies. It covers two common isolation setups:
- Air-gapped offline environments: The host has no internet access or forbids all external network traffic. Inference, tool execution, workspace edits, and search stay completely local.
- Self-hosted intranet environments: Agents may reach approved internal services (such as self-hosted Ollama/vLLM, SearXNG, GitLab CE, or GlitchTip), while blocking public cloud model fallback and unlisted internet hosts.
You first configure the daemon's baseline model lock and network policy on the host, then drive repository attachment, AGENTS.md boundaries, and end-to-end verification in natural language through the Web GUI.
Prerequisites
- A working Holon installation with permission to edit
<HOLON_HOME>/config.jsonand restart the daemon. - A running local model server:
- Local or intranet Ollama (default
http://127.0.0.1:11434) or an OpenAI Chat Completions-compatible vLLM server (defaulthttp://127.0.0.1:8000/v1), with your target weights already downloaded. - If you run multi-turn coding agents on Apple Silicon or a 64GB unified-memory workstation, see Can Local LLMs Complete Multi-Turn Coding Agent Tasks? Benchmarking GGUF, MoE Turn Bloat, and Prompt Caching for measured recommendations (for example,
qwen3.8:27b-gguforgemma4:26b, and writing aModelfileon disk instead of pipingollama create -f -).
- Local or intranet Ollama (default
- Optional intranet services (self-hosted mode only): Docker host for SearXNG, an internal Git server, or a local documentation search script.
Step 1: Lock Local Models and Disable Cloud Fallback
Setting model.default to a local model is not enough on its own. If cloud API keys exist in the daemon's environment and provider fallback remains enabled, a local timeout or error can fail over to a cloud provider.
Before starting or restarting the Holon daemon, lock the default model and disable fallback on the host:
Option A: Local Ollama (No API Key Required)
holon config set model.default "ollama@default/qwen3.8:27b-gguf"
holon config set runtime.disable_provider_fallback true
If you pull a GGUF tag directly from the Ollama library (avoiding default MLX variants on Apple Silicon), you can also use routes such as ollama@default/qwen3.8:27b-q4_K_M or ollama@default/gemma4:26b.
Option B: Local or Intranet vLLM (No API Key Required)
# When vLLM serves at default http://127.0.0.1:8000/v1
holon config set model.default "vllm@default/<your-served-model>"
holon config set runtime.disable_provider_fallback true
If Ollama or vLLM runs on another GPU server on your LAN—or if Holon runs inside a container and reaches the host—update the provider's base_url:
# Example: point Ollama to an intranet GPU server
holon config providers set ollama --transport anthropic_messages --base-url "http://10.0.10.20:11434" --credential-source none --credential-kind none
Tip: Leave cloud
vision.default,image_generation.default, andx_searchunconfigured. To ensure no cloud routes are ever eligible, unset any leftoverOPENAI_API_KEY,ANTHROPIC_API_KEY,DEEPSEEK_API_KEY, orXAI_API_KEYvariables before launching the daemon.
Step 2: Configure Web and Search Policy for Your Boundary
Holon gates WebFetch, WebSearch, and XSearch at the runtime level. Pick the mode that matches your environment.
Mode A: Air-Gapped Offline — Disable External Web Tools
In a disconnected environment, turn off public web fetching and search so agents do not waste turns waiting for network timeouts:
holon config set web.fetch.enabled false
holon config set web.search.enabled false
holon config set x_search.enabled false
If you still want agents to use WebSearch against a local offline knowledge base (such as local Markdown wikis or offline documentation indexes), keep web.search.enabled true and configure a kind=command search provider that executes purely on disk without HTTP requests:
{
"web": {
"search": {
"enabled": true,
"provider": "local_docs",
"mode": "single"
},
"providers": {
"local_docs": {
"kind": "command",
"command": {
"argv": ["/usr/local/bin/search-local-docs", "--query", "{{query}}", "--limit", "{{max_results}}"]
},
"output": {
"format": "json",
"mapping": {
"title": "$.results[*].title",
"url": "$.results[*].url",
"snippet": "$.results[*].snippet"
}
}
}
}
}
}
Mode B: Self-Hosted Intranet — Private Search and Strict Fetch Allowlist
In a self-hosted intranet setup, you want agents to reach internal services (such as SearXNG, GitLab CE, GlitchTip, or internal docs) while blocking public internet egress:
-
Connect self-hosted SearXNG for
WebSearch: Follow Set Up Self-Hosted SearXNG as Holon WebSearch Provider, then make your local instance the default and first automatic candidate:holon config set web.providers.searxng.kind searxng holon config set web.providers.searxng.base_url "http://127.0.0.1:8888" holon config set web.search.enabled true holon config set web.search.provider searxng holon config set web.search.providers '["searxng"]' holon config set web.search.mode single holon config set web.search.max_provider_attempts 1 holon config set web.search.builtin_provider.enabled false holon config set x_search.enabled falseThe attempt cap also applies to explicit aggregation. It is not a provider allowlist: use network egress controls to enforce the intranet boundary.
-
Restrict
WebFetchwithweb.fetch.allowed_hosts: By default,WebFetchblocks loopback (127.0.0.1,[::1]) and private network ranges (10.0.0.0/8,192.168.0.0/16, etc.) to prevent SSRF. Setting a non-emptyweb.fetch.allowed_hostslist serves two purposes at once: it explicitly permits those listed internal hosts/ports while blocking every unlisted public internet host:holon config set web.fetch.enabled true holon config set web.fetch.allowed_hosts '["127.0.0.1:8080", "git.internal.corp", "docs.internal.corp"]'Cloud metadata IPs (such as
169.254.169.254) remain unconditionally blocked even if added toallowed_hosts.
Step 3: Lock Down Decision and Observability Subsystems
-
Advisory Decision (
AdvisoryDecision): Keepdecision.enabledset tofalseif unused. If you need advisory decisions in an air-gapped environment, use embedded local ONNX (local-onnx) pointing at a pre-staged directory on disk rather than downloading presets at runtime:holon config unset decision.model holon config set decision.local_onnx.model_dir "/opt/holon/models/jev-selector-q4f16" holon config set decision.local_onnx.enabled true holon config set decision.enabled trueSee Configure Advisory Decisions for details.
-
Observability (OpenTelemetry): OTLP trace export is disabled by default (
runtime.observability.otlp.enabled = false). For centralized intranet telemetry, point the endpoint strictly to a localhost or LAN collector (e.g.,http://127.0.0.1:4318/v1/traces); see Configure Observability.
Restart the Holon daemon to apply host changes and verify configuration health:
holon config doctor
Step 4: Attach Workspaces and Set Offline Toolchain Rules in Chat
Once the daemon is running, handle repository attachment, isolation boundaries, and standing toolchain rules directly in Web GUI chat.
1. Attach a Local Repository and Enforce Worktree Isolation
Tell your agent in chat:
"Attach
/srv/repos/internal-serviceas a workspace. Create a dedicated Git worktree frommainfor all upcoming edits and test runs, and do not modify the main checkout."
The agent calls AttachWorkspace and CreateWorktree to isolate edits and test commands. See Work in a Project Workspace or Git Worktree for safety checks and cleanup behavior.
2. Record Standing Offline Toolchain Rules in AGENTS.md
To make local-only toolchain rules persistent across every wake and session, instruct the agent to write them into its standing contract (AGENTS.md):
"Record these standing rules in your
AGENTS.md:
- You are operating in an offline/intranet private environment. Never run
curl,git clone,npm install,pip install, orcargocommands that attempt to reach public internet registries.- Run builds and tests using pre-cached local dependencies and offline flags (such as
cargo test --offlineornpm ci --offline).- Base every delivery report strictly on local files and observed test exit codes."
The agent persists these constraints in agent_home/AGENTS.md (or the repository's AGENTS.md), so they load automatically on every future turn.
3. Optional: Connect Self-Hosted Team Services
For a full intranet team workflow without public cloud accounts, pair Holon with self-hosted identity and event sources:
- Self-hosted SSO: GitLab CE OIDC, Keycloak OIDC, or Casdoor OIDC.
- Internal error and web-change webhooks: Self-Hosted GlitchTip Error Alerts or Self-Hosted changedetection.io.
Step 5: Verify Boundaries and Inspect Audit Receipts in Chat
Before handing off production repositories, run one verification prompt in Web GUI chat to check model routing, network blocking, and local execution:
"Run a private-environment self-check without modifying any source files:
- Report your current
model.requested_model,model.active_model, andmodel.fallback_active.- Try calling
WebFetchonhttps://example.comand report whether it is blocked and the exact error.- If an internal allowlisted service or local search is configured, run one allowed check; then run a minimal read-only test command in the attached workspace and report the exit status."
Confirm the returned evidence:
- No cloud model egress:
model.active_modelmatches your localollama@default/...orvllm@default/...route, andmodel.fallback_activeisfalse. - Public internet blocked:
- In air-gapped mode,
WebFetchandWebSearchare disabled or rejected. - In intranet allowlist mode,
https://example.comfails withnetwork_denied(not inweb.fetch.allowed_hosts), while explicitly allowlisted internal hosts succeed.
- In air-gapped mode,
- Auditable local receipts: Inspect the task details and tool execution log in the Web GUI (see Observe, Track, and Verify Agent Progress) to verify local command exit codes and confirm zero unintended external calls.
Troubleshooting
WebFetchblocks localhost or LAN hosts withnetwork_denied: Loopback and private IPs are blocked by default. Add the exact hostname orIP:port(e.g.,"127.0.0.1:8080"or"git.internal.corp") toweb.fetch.allowed_hostsand restart the daemon.- Holon cannot reach a running local Ollama server:
Check
curl http://127.0.0.1:11434/api/tagson the host. If Holon runs inside a container while Ollama runs on the host, configure Ollama'sbase_urltohttp://host.docker.internal:11434(or the host's LAN IP) and verify Ollama listens on that interface. - Multi-turn coding tasks time out on local hardware:
On 64GB Apple Silicon running 27B models, confirm Ollama is not running the MLX variant with slow first-turn prefill. Switch to
ggml-orgQ4_K_MGGUF (via an explicitModelfileon disk) and keep default FP16 KV caching; see the multi-turn local benchmark. AdvisoryDecisionreports missing presets offline: Do not trigger runtime preset downloads without internet access. Copy the verified model directory to disk, setdecision.local_onnx.model_dir, and unsetdecision.model.
See Also
- Can Local LLMs Complete Multi-Turn Coding Agent Tasks? Benchmarking GGUF, MoE Turn Bloat, and Prompt Caching
- Configure and Switch Models: From Local Ollama to Multi-Cloud Failover
- Set Up Self-Hosted SearXNG as Holon WebSearch Provider
- Fetch and Search Web Content
- Work in a Project Workspace or Git Worktree
- Configuration Reference
