How it works · Coalent

Coalent has a small number of moving parts. Let's meet them in plain language, then see how they fit together.

The four pieces

Retriever — how Coalent fetches your information. You point it at wherever your knowledge lives — a vector database, a set of documents, an API, or a tool. Given a question, it returns the relevant raw pieces (each one is a Chunk). This is the only part most teams write, and it's a single method. → The Retriever

Synthesizer — how raw information becomes understanding. It takes those raw chunks and produces a structured understanding using your LLM. As of v0.4 that understanding is extractive: a query-independent list of atomic, source-grounded claims — not a question-shaped prose summary. Because it isn't written to answer one specific question, a single unit answers many later ones, and it keeps every number (prose summaries dropped ~40% of them in testing). It also notes which sources it actually used. → The Synthesizer

Cognition unit — one cached understanding. The result of the above: a small briefing built from one slice of your knowledge. It keeps the understanding — the list of claims — the raw evidence it was built from, and its sources (so Coalent knows what it depends on). Units are lightweight and independent, built lazily only when a query needs one — the opposite of paying to build a whole graph upfront.

Invalidation — keeping it correct. When one of those sources changes, Coalent marks only the cognition units that used it as stale. They rebuild on the next read — nothing else is touched. → Provenance & freshness

The failure chain — getting better on failure. New in v0.7 When a read fails by your app's own judgment, the cache doesn't just retry — it heals: repair(read_id) re-extracts the facts the build missed and appends them permanently, so the same failure never costs you twice. Two more $0 rungs (reprobe, serve_unserved) recover answers that were in the store but buried. All default-OFF, all driven by your evaluator, never the library's. → The failure chain

The big picture

You askcache.get("why is the search service slow?")

CognitiveCache

1
Embed & match
Embed the query; reuse a cached unit when the meaning matches.
2
Retriever
On a miss, fetch the evidence for the query (your retrieval).
your vector DB · GraphRAG · tools · APIs
3
Synthesizer
Build structured understanding and cite the sources it used.
your LLM
4
Cache the unit
Store understanding + retained raw + provenance.
memory · SQLite
5
Return
Hand back the minimum decision-relevant context (raw on tap).

Source changeingestion · webhook · CDC · TTL revalidate

A change marks only the cognition units that used that source stale (via provenance). The next matching read rebuilds just those, lazily — nothing else.

First time you ask (a miss / below-gate read): Coalent retrieves, synthesizes a cognition unit, and caches it.
Ask something similar (a hit): it serves the cached understanding instantly — matched by meaning, so a rephrase still hits.
Ask something that spans two documents (multi-hop): on the default pool path this needs no mechanism — every read ranks the claims of all fresh units in one pool, so cross-document evidence competes by construction. (The legacy unit path covers it with cross-unit recall — a sweep over all fresh units' claims at zero extra LLM calls.)
A source changes: only the units that used it rebuild, lazily, on the next read — and their claims are masked from the pool instantly.

TIP

Since v0.7.0 the read path defaults to pool whenever a semantic embedder is available — pool ranking + an adaptive serve gate, the path every benchmark number was measured on. Keyless construction falls back to the legacy unit path (a ladder of gates: match, freshness, coverage, recall, the RAG floor) with a warning; explicit read_path="unit" is byte-identical to pre-0.7 behavior. See The gate ladder for both firing orders, and Context intelligence for the pool payload.

The read, end to end: serve lane and trouble chain

New in v0.7 On the default pool path, the full pipeline splits into two lanes with very different lives. The serve lane runs on every read and never calls an LLM: the query (plus your planner's optional sub-questions) becomes a probe union, every probe scores the global fresh-claim pool, the winners pack into the serve budget, and the Result ships with its own signals — read_id, confidence, coverage, gaps, needs_retrieval. The trouble chain exists only past your app's own evaluator gate: repair converts the read's banked candidate spans into permanent claims, a second get() lets them compete, and on a persistent refusal reprobe and serve_unserved re-work the unchanged pool at $0. Because a correct answer never enters the chain, it structurally cannot break one — and because repair writes back to the store, a failure is paid once, then every future read is first-pass.

SERVE LANE — every read · no LLM TROUBLE CHAIN — only after YOUR evaluator says failed app territory

cache.get(query, subs=, constraints=)

Probe union

raw query + ≤4 sub-questions · ONE batched embed · subs=None → raw query only

Global claim-pool rank

every probe × every fresh claim · score = MAX cosine

⇢

Gap detector

gap_detector=True · evidence sentence outscores every claim on a probe

⤳ banks the span on the read's ledger + reports it in Result.gaps — observes, never packs (pinned)

⇢

Constraints

constraints={dates·sources·entities} vs unit ingest metadata

⤳ matched units' best spans join the ledger — feeder only, zero serving delta (pinned)

Pack to serve_budget

rank-prefix walk · attribution headers counted

Result — the payload + its signals

$0 advisory INPUTS to your failure logic — never actions

Your answerer

your model, your prompt

Your evaluator — did the answer fail?

YOUR contract: answer flag · judge node · our signals

↳ good

Done — repair never fires

a successful read pays nothing new

repair(read_id) — the pumpBYO extractor

ledger candidates, then bridge near what served · span-anchored BYO extractor · hygiene + dedup · append ≤8 claims + per-claim provenance

Second get()

repaired claims compete in the pool like any claim

reprobe(read_id)

entity probes from SERVED claims · MAX-union re-rank — nothing served can rank worse · embeds only, no LLM

serve_unserved(read_id)

force-pack admitted-but-unserved repaired claims + the best constraint-matched unit absent from the payload · one embed, no store writes

answer again — finalworst case: an honest refusal, gaps reported

THE WRITE-BACK LOOP — PERMANENT

repair appends to the store, once. The claims carry per-claim provenance (traceable, removable), and every future read of every user ranks them in the ordinary serve lane — the cost is paid once, the fix is first-pass forever. A pipeline that misses a fact misses it identically every time; this cache gets better every time it fails.

rung safety, structural: repair is gated on a failed read; reprobe / serve_unserved run on refusals — a refusal is never a correct answer, so the downstream rungs have nothing to break.

corpus_hole gaps route to YOUR tool nodes (fetch → ingest → retry) — the cache never fetches on its own authority. report_refusal(read_id), the v0.6 behavioral retry, composes alongside the chain.

Activation conditions sit on every arrow: the taps bank candidates but never touch serving (pinned byte-identical), the chain fires only on your evaluator's "failed", and the write-back arrow is the thesis — the cache gets better every time it fails. The mechanics of every rung: The failure chain; where it slots into the failure handling you already run: Integrating with your failure handling.

Matched by meaning

You never label your questions or configure routing. Coalent matches a question to cached understanding by its embedding — so "how much annual leave?" and "what's the leave allowance?" land on the same unit. The cache organizes itself around what questions mean.

Multi-hop, for free

Some questions can't be answered from any single unit — the answer is split across two documents. On the default pool path this needs no mechanism at all: the cache serves a token-budgeted pool of globally-ranked fresh claims across all units, so cross-document evidence competes in one ranking by construction, with stale units' claims masked the moment a source changes — units remain the ownership / freshness / build / provenance skeleton. Measured: naive's best measured accuracy at 25–43% fewer context tokens (n=605, strict grading). See Context intelligence.

On the legacy unit path (read_path="unit"), multi-hop is handled by cross-unit claim recall New in v0.4: when the matched unit under-covers the question, it sweeps the atomic claims from every fresh unit (a MaxSim over their per-claim memory) and surfaces the ones that bridge the gap — zero extra LLM calls, surfaced as result.recalled. preset="multi_hop" New in v0.5 arms it plus a hop-2 bridge in one argument. There is no recall step on the pool path — the scan is already cross-unit, and result.recalled stays empty there.

Decision-ready, not a data dump

A cognition unit isn't a pile of chunks — it's distilled, structured understanding, and on each read Coalent hands back the minimum payload relevant to your question: the budget-packed, attributed claim pool on the default path (raw reachable via drill, or appended automatically when a read under-covers), or the matched unit's query-relevant slice on the legacy unit path. Less noise to the LLM means better answers and fewer tokens. → Context intelligence

The Retriever — connect your data.
The Synthesizer — build understanding with your LLM.
The gate ladder — the full read path, gate by gate.
The failure chain — the self-healing loop, rung by rung.
Examples — wire it to real systems.