CyberCLI
Sign in →

← All recipes · AI

Wire the local AI lane via Ollama

~15 minutes

Run the entire AI digest path on a local Ollama server — no API costs, no data egress, full sovereignty.

Last updated 2026-06-07 · 4 steps

§1

Prerequisites

§2

What you'll do

You'll pull a Cyber-Guild-suitable model into Ollama, configure CyberCLI's AI lane to local (or byo_server if Ollama is on a sibling host), benchmark the throughput with `cybercli doctor`, and watch the first digest emit with `parameters.__provider_attribution__` pointing at your Ollama.

§3

Steps

Step 1

Pull a model into Ollama

The Warden role's first-pass triage works well with llama3.1:8b-instruct or qwen2.5:7b. Either is around 4-5 GB on disk and runs at 50+ tok/s on a modern consumer GPU.

For Pro tier installs the v1.x roadmap adds Cyber Guild fine-tuned models that ship via Ollama too; the wiring is the same.

$ ollama pull llama3.1:8b
Verify

`ollama list` shows llama3.1:8b with size + last-modified timestamp.

Step 2

Configure the AI lane

From the CyberCLI dashboard, open /settings → AI engine. If Ollama is on the same host, pick lane=local + model=llama3.1:8b. If Ollama is on a sibling, pick lane=byo_server, paste the Ollama URL (e.g. `http://hermes01:11434`), and pick the model.

Save. The engine writes the AiPolicyRecord; the next event will route through Ollama.

Verify

/settings shows the chosen lane + model + 'healthy' indicator. The audit chain advances with an `ai_policy_set` row.

Step 3

Benchmark the throughput

Run `cybercli doctor`. The doctor's local AI benchmark check runs a short prompt against your Ollama endpoint and reports tok/s. Under 50 tok/s suggests CPU-only inference or thermal throttling; under 10 tok/s suggests serious headroom issues.

The hardware advisory in /settings flags low VRAM if it detects an undersized GPU; respect the advisory.

$ cybercli doctor
Verify

Doctor prints `local AI benchmark · X tok/s on llama3.1:8b — healthy` where X is comfortably above 30.

Step 4

Watch the first local digest

Trigger a new event into the pipeline (any connector poll will do — Wazuh's next 60-second poll, or generate a syslog test event).

Open /audit and filter by outcome=`digest_emitted`. The most recent row's `parameters.__provider_attribution__` should now show `provider_name: local_ollama` + `model: llama3.1:8b`. That's the smoking gun: your AI now runs locally.

Verify

/audit's latest digest_emitted row carries `parameters.__provider_attribution__.provider_name: local_ollama`. /signals' newest row carries an AI-derived disposition.

§4

Verify end to end

§5

Troubleshoot

Doctor reports `connection refused` for the Ollama benchmark.

Cause Ollama is not running, OR is listening only on 127.0.0.1 while CyberCLI is on a sibling host.

Fix On the Ollama host: `systemctl status ollama`. If it's listening only locally, set `OLLAMA_HOST=0.0.0.0:11434` in the systemd unit env, restart, and re-test.

Doctor reports tok/s < 10 — Ollama is healthy but slow.

Cause Likely CPU-only inference (no GPU detected) or thermal throttling under sustained load.

Fix Verify GPU detection: `nvidia-smi` (or `radeontop` for AMD). If the GPU is present but unused, install/restart the right CUDA / ROCm driver; if no GPU, lane=local will be slow — consider byo_server pointed at a GPU host.

`ai_policy_set` audit row shows lane=safe_disabled instead of local.

Cause The license tier validation rejected lane=local (rare) OR the configure call errored.

Fix Free tier supports lane=local; if you see this on Free, run `cybercli doctor` and check the configuration error message. The dashboard's /settings page will show the underlying error.

What to do next

Install CyberCLI from scratch

From a fresh Ubuntu VM (or LXC) to a running engine with the dashboard open at /first-30-minutes, in about 8 minutes.
Connect your first Wazuh manager

Point CyberCLI at an existing Wazuh manager, watch the first alerts flow through the SOC pipeline, and verify the audit chain advances.