CyberCLI

← All engines · AI

DigestPipeline

stable

v1.0.0 · last updated 2026-06-06 · source: cybercli.digest.pipeline.DigestPipeline

§1

What it does

Routes each canonical event through a two-pass AI flow — a cheap gate that picks the right Guild role + prompt template, then a per-role provider that produces the structured Digest the bridge gates on.

§2

Why you care

The DigestPipeline is the engine that turns a normalized event into a structured AI decision. The first pass is a tiny gate call that asks 'which Guild role should look at this and which prompt template should it use?' — Warden for triage, Knight for verification, Marshal for sequencing, and so on. The second pass is the real digest — the role's actual reasoning under its charter, against the operator's evidence, producing a Digest the bridge can gate on.

Splitting the call in two costs one cheap gate token-pass but saves the expensive deep-reasoning call from running with the wrong template on events that don't need it. On a Free-tier install with local Ollama, that's the difference between an llama3.1:8b call that costs nothing and a frontier-tier API call that costs real money on a Pro install — without the gate, you'd pay frontier-tier rates on every 4xx that should have been triaged-and-discarded.

The pipeline is also where the per-role provider routing happens. A Pro buyer can wire Warden to Anthropic, Knight to local Ollama, and Marshal to OpenAI — different model strengths matched to different cognitive tasks. The per-role TOML override (`[digest.providers.<role>]` in config.toml) is read on construction and the routing table fires deterministically per event.

§3

How it works

On each event the pipeline first runs `_run_gate_pass()` — a short prompt that asks the gate provider to return a `GateRoute` object naming the role to invoke + the prompt template version. The gate provider is configurable; defaults to the same lane as the digest provider but operators routinely point it at a smaller / cheaper model since the gate prompt is small and structured.

With the gate's verdict in hand, the pipeline picks the role's provider from `self.role_providers` (the per-role override map) or falls back to `self.provider`. It then builds an isolated prompt — the operator's untrusted artifact wrapped in sentinels, with the system instructions + expected JSON schema as a separate block — and dispatches to the provider's `generate()` method.

The provider returns a `RawProviderResponse` carrying the raw model text + provider metadata (model, prompt hash, eval count, durations, content hash). The pipeline parses + validates against the `Digest` Pydantic schema — schema mismatches are surfaced as `DigestPipelineError` with the raw output for the operator to inspect.

After validation the pipeline runs framework drift correction: if the model hallucinated the prompt_template_version, model_route, event_ref, or evidence artifact_refs, the framework rewrites them to the deterministic values and logs the drift. Local models drift constantly (they were trained on Claude-flavored outputs); the framework treats drift as expected for local and as worth-a-WARN for API providers. The corrected Digest is what flows into the bridge.

§4

Configure & observe

CLI

$ cybercli smoke --only ollama

Probe the configured Ollama endpoint + verify the model is loaded. The first thing to run when the digest pipeline silently routes to human-only.

$ cybercli doctor

Runs the digest-provider wiring check + the local AI benchmark (tok/s on the chosen model). Tells you up-front whether your hardware can sustain the configured pipeline.

Dashboard surfaces

/ai-agents

The Guild roster + per-role provider configurator. Each role card carries its current provider, recommended model for the tier, and an inline Configure sheet.

/settings (AI engine card)

The lane configurator (local / byo_server / api / off) + hardware advisory that fires when local lane is picked on a CPU-only / low-VRAM host.

/audit (filter outcome=approved_autonomous)

Every row carries `parameters.__provider_attribution__` with the model that fired the digest — read forensically to answer 'which model said what.'

Audit row shape

Outcomes emitted
digest_emitteddigest_provider_errordigest_validation_failed
Parameters block

parameters.__provider_attribution__ carries model, provider_name (e.g. local_ollama, AnthropicProvider), chosen_provider_role (Warden / Knight / etc.), prompt_eval_count, eval_count. The full Digest body is attached as parameters.digest.

§5

Status

Stable — the two-pass routing has been on the v1 acceptance install since 2026-06-06 and has produced 19 triage decisions across 51 signals. The schema validator is strict (`extra=forbid`) so a model that adds an unexpected field gets caught at the boundary.

Known limit: only the triage purpose (Warden's first pass) fires in v1. Multi-pass orchestration where Knight verifies a Warden finding, Marshal sequences a response, and Paladin executes the chain ships in v1.1. The per-role provider override map is wired today so a Pro buyer can pre-stage Knight + Marshal + Paladin providers ready for v1.1; they just don't fire until the orchestrator lands.

Shipped 2026-06-07: runtime now wires `DigestPipeline.gate_provider` from a `[digest.gate]` config block. Common Pro-tier pattern unlocked: gate on local Ollama (cheap + structured), digest on Anthropic (frontier reasoning). The gate config is operator-pinned (the AiPolicyRecord override applies only to the deep digest lane), so a 'dashboard says api lane' install can still pin its gate to local for cost.

How this engine talks to the rest

Upstream · feeds this engine
Downstream · this engine feeds