The failure chain

New in v0.7 v0.7 gives the cache an explicit failure chain your agent drives. The design principle behind every surface on this page: your failure structure stays in charge — Coalent adds rungs and signals to it. The library never string-matches your answers, never decides "failed" on its own, and never fetches on its own authority. Your app's existing evaluator — a structured answer flag, a judge node, or the $0 signals below — makes the call; the cache supplies the verbs.

Three things to learn, in this order:

1. Signals — $0, advisory, on every read: confidence, coverage, needs_retrieval, gaps, read_id. Inputs to your failure logic, never actions.
2. Verbs — what your app calls when its evaluator says failed: repair(read_id) → a second get() → reprobe(read_id) → serve_unserved(read_id), plus v0.6's report_refusal(read_id). That order is the recommended ladder on a persistent refusal — but each verb is independent and safe to call alone.
3. Guarantees — each one a pinned test, not a promise (the table at the bottom).

Every knob and verb on this page is default-OFF and byte-inert until armed (pinned by dedicated inertness tests). v0.7.0's one deliberate default change lives elsewhere: since 0.7.0 the read path defaults to pool whenever a semantic embedder is available — the measured path behind every benchmark number — with keyless construction falling back to the unit path, and explicit read_path="unit" byte-identical to pre-0.7 behavior (the gate ladder). The chain runs on the pool path — arming a chain knob on the legacy unit path fails loud (ValueError) rather than sitting silently inert. (The two always-on Result additions — sources and max_source_age_s — ride on both read paths.) And the chain adds cost only on failed reads: a read that succeeds pays nothing new.

Measured on the same frozen news rig as every anchor since v0.5 (609 articles, 605 held-out questions, strict grading + locked adjudication rules): the full v0.7 composition scores 0.826 vs 0.774 for the strongest v0.6 configuration — +5.3 points at an identical ~983-token serving budget — and final refusals fell 69% (61 → 19). Full numbers and framing on the benchmark page.

The signals — the read ships its own doubt

Every read already returns read_id, confidence, coverage, and needs_retrieval (get() & Result). New in v0.7 adds provenance and doubt to the same surface, so an evaluator node can act without drilling:

r = cache.get(question)
r.sources             # artifact ids actually behind the served payload (both read paths)
r.max_source_age_s    # freshness age of THIS serve — MAX served-owner age, seconds
r.probes              # the probe texts this read scored (raw query first)
r.probe_coverage      # per probe: {probe, best_claim, best_span, margin, fired}
r.gaps                # the actionable subset — see the gap detector below
r.parent_read_id      # set on reprobe()/serve_unserved() results; "" on ordinary reads

These are inputs to your failure logic, never actions. A non-empty gaps is not an error — it's the read saying "here is where I doubt myself", for your evaluator to weigh.

gap_detector=True — observe-only, on every read

What it does. Per probe, if a raw evidence sentence outscores every fresh claim by a fixed margin, that sentence is banked on the read's ledger as a repair candidate and reported in Result.gaps. Each gap is typed:

When it activates. Constructor knob gap_detector=True (pool path — the default in keyed deployments; ValueError on the legacy unit path). Once armed it observes on every read; a gap fires only when an evidence sentence actually outscores every claim on some probe.

What it costs. $0 at serve: the evidence-sentence tier hydrates lazily per unit, embeds once, and is cached. No LLM, ever.

What it cannot break. Observe-only by construction: serving is byte-identical ON vs OFF (pinned). It is also the one signal that fires without a refusal — catching the confident-wrong class that refusal-gated machinery is structurally blind to.

cache = SemanticCache(retriever, synth, embedder=OpenAIEmbedder(),
                      gap_detector=True)   # pool path — the default with a semantic embedder (v0.7)

r = cache.get(question)
for gap in r.gaps:                     # {probe, span, unit_id, source, kind}
    if gap["kind"] == "extraction_hole":
        ...                            # repair terrain — see repair() below
    else:                              # "corpus_hole"
        my_fetch_tool(gap["probe"])    # YOUR tool, YOUR ingestion, then retry

subs= — planner-owned decomposition

What it does. Your planner hands the read its decomposition: subs=["...", ...] (strings, or {"q", "hyde"} dicts) become probes alongside the raw query — one batched embed, a claim's score = its best across probes (the probe union). This is the one v0.7 surface that changes first-pass ranking too, which is why it's the accuracy-max option and caller-owned: the cache never guesses a decomposition.

When it activates. Only when you pass it (pool path — the default in keyed deployments; ValueError on the legacy unit path). subs=None (default) → raw query only, first-pass ranking unchanged. subs=[] explicitly sanctions no decomposition. Malformed entries fail open to the raw query. Up to 4 sub-questions join the union. An explicit subs= wins over the decompose= constructor callable (below) for that read.

What it costs. One batched embed call. No LLM.

The risk decision, explicitly. This is the fork between the two adoption modes: zero-risk mode passes nothing — the first pass stays byte-identical to v0.6 and the chain engages only on failure; accuracy-max mode passes subs= and accepts first-pass churn for the measured 0.826 composition. Both compositions ship; subs= is caller-owned, never guessed. See the adoption modes.

r = cache.get(
    "How did the two chip announcements this week affect the Nvidia supply chain?",
    subs=[                                # from YOUR planner — never guessed
        "What chip announcements happened this week?",
        "What is the Nvidia supply chain exposure?",
    ],
)

For naked deployments without a planner there is decompose= — a BYO constructor callable (query) -> [{"q", "hyde"}, ...] whose sub-questions join the same probe union (clamped at 4; default OFF; the library never calls an LLM itself).

constraints= — intent metadata, feeder only

What it does. constraints={"dates": [...], "sources": [...], "entities": [...]} — intent detection's natural output — is matched against unit ingest metadata (meta= at ingest, see below): dates through one ISO canonicalizer (never guessed), sources/entities as case-insensitive substring. Matched units' best evidence sentences join the read's repair-candidate ledger.

When it activates. Only when you pass explicit values (pool path — the default in keyed deployments). No values, or no match → a clean no-op. With ≥2 derivable keys the match is AND across keys / OR within a key's values; an empty AND set falls back to OR (with a constraints_and_fallback event) — the conjunction can narrow but never silence a read's candidates.

What it costs. A metadata comparison — $0. No LLM, no embeds.

What it cannot break. Feeder only (pinned): constraints never touch pool scoring or serving — zero serving delta, ever. They only make repair() smarter about where to look, and give serve_unserved() its second candidate source.

r = cache.get(
    "What did TechCrunch report on October 5?",
    constraints={"dates": ["2023-10-05"], "sources": ["TechCrunch"]},
)
# serving is unchanged; the matched units' best spans are banked for repair(r.read_id)

The verbs — your evaluator decides, the cache heals

The wiring pattern (also the worked recipes):

cache = SemanticCache(
    retriever, synth, embedder=OpenAIEmbedder(),
    read_path="pool",
    gap_detector=True,               # observe-only: the read reports its own holes
    repair_extractor=my_extractor,   # BYO callable(span, region, existing_claims) -> [claims]
)

r = cache.get(question, subs=planner_subquestions)
answer = my_answerer(r.context["pool"])          # your model, your prompt

if failed(answer):                               # YOUR evaluator — the chain's only gate
    cache.repair(r.read_id)                      # re-extract what the build missed — PERMANENT
    r2 = cache.get(question, subs=planner_subquestions)
    answer = my_answerer(r2.context["pool"])     # repaired claims now compete
    if failed(answer):
        r3 = cache.reprobe(r2.read_id)           # entity-probe re-rank of the same pool
        answer = my_answerer(r3.context["pool"]) if r3 else answer
    if failed(answer):
        r4 = cache.serve_unserved(r2.read_id)    # force-pack what the chain located
        answer = my_answerer(r4.context["pool"]) if r4 else answer

repair(read_id) — the pump

What it does. Converts a failed read's candidate spans into permanent pool claims, so the fact the extractor missed serves at the next first pass through the unchanged ranker. Candidates = the read's banked ledger (gap-detector fires + constraints matches, best score first), then bridge candidates derived from the top served claims — holes near what served, which is why the queue is never empty even when the detector was blind. Per candidate, one span-anchored call to your repair_extractor(span_text, context_region, existing_claims) — the span, a bounded ±1-sentence region, and the owning unit's current claims as the do-not-repeat list. Survivors of admission hygiene and dedup append to the owning unit with per-claim provenance. Returns a RepairReport.

When it activates. An explicit app call, only — it never fires on its own, and it's entirely off the read path. Requires repair_extractor= at construction (RuntimeError otherwise; the library never calls an LLM itself — the extractor is your model, your key). An unknown or aged-out read_id, or a read with nothing banked and nothing served, is a clean no-op report.

What it costs. The extractor calls (BYO spend) — and only on reads your evaluator failed. Measured: gating repair on failure matches always-on accuracy at 14% of the extraction calls; calling it on successful reads is measured waste.

What it cannot break — and what makes it permanent.

• Append-only: existing claims are never rewritten or dropped.
• Per-claim provenance: every admitted claim records {claim, unit_id, span, source, origin, via, ts} under understanding["_repair_provenance"] — a wrong-but-novel claim stays traceable and evictable by inspection.
• Admission hygiene rejects the two measured defect shapes mechanically, before dedup: antecedent-free pronoun-subject claims ("He was the richest…" with no proper noun) and truncated claims (mid-sentence starts, dangling connectors). Counted in RepairReport.rejected, never silently dropped.
• Dedup: exact-normalized match or cosine ≥ 0.95 against the unit's claims — a rephrasing of something already known is refused.
• Capped at 8 admissions per call; stale or misaligned units are skipped entirely.
• The improvement persists through the normal store path — the cost is paid once, every future read of every user benefits.

report = cache.repair(r.read_id)
report.candidates_seen   # candidate spans considered (ledger + bridge, deduplicated)
report.extracted         # claim strings your extractor returned
report.rejected          # refused by admission hygiene (defect shapes)
report.admitted          # appended to their owning units (≤ 8 per call)
report.claims            # one provenance dict per admitted claim
report.units_touched     # which units grew

The repair_extractor contract — any text-in/text-out model works:

def my_extractor(span: str, region: str, existing_claims: list[str]) -> list[str]:
    """Extract atomic claims from `span` (with `region` for context),
    skipping anything already stated in `existing_claims`."""
    out = my_llm(EXTRACT_PROMPT.format(span=span, region=region, known=existing_claims))
    return parse_claim_list(out)

reprobe(read_id, hint=None) — the buried-answer rung

What it does. A mechanical second pass over the unchanged pool for answers that are in the store but buried: harvests proper-noun entities from the read's served claims (question tokens excluded), pairs them with the read's sub-question tails as new probes ("<entity> — <tail>"), embeds everything in one batched call, and re-ranks under the MAX-union of the original probes plus the new ones. The union always includes the originals, so nothing already served can rank worse. Returns a fresh Result with a new read_id and parent_read_id set. An optional hint (e.g. your answerer's draft) joins as one extra probe.

When it activates. An explicit app call; the recommended contract is on a persistent refusal after repair + the second read. Self-skips (returns None, with a reprobe_skipped event) on an unknown/aged-out read_id, when the served claims carry no harvestable entities, or when the probe embed fails — it never crashes your loop.

What it costs. One embed batch. No LLM, no retrieval, no build, no store mutation.

What it cannot break. Two structural properties: the MAX-union guarantee (served claims can only hold or improve their rank), and the refusal contract — called on refusals, and a refusal is never a correct answer, so there is nothing to un-answer.

serve_unserved(read_id) — the last $0 rung

What it does. Force-packs, at the head of a fresh payload: (a) this question's repaired claims that were admitted but never served — the chain found them, the packing race dropped them; (b) the top claims of the highest-scoring constraint-matched unit absent from the served payload. The read's original served claims then refill the remaining budget in served order, so cross-article comparison context survives. Fresh Result, new read_id, parent_read_id set.

When it activates. An explicit app call — the rung after reprobe on a persistent refusal. Pass the chain's latest ordinary get() read id (the post-repair second read). With no unserved candidates it emits serve_unserved_skipped and returns None — it only ever fires meaningfully.

What it costs. One embed(query) call. No LLM, no retrieval, no store mutation.

What it cannot break. By contract it runs on refusals — a refusal is never a correct answer, so like reprobe it cannot break one. It never mutates the store; the banked ledger is read non-destructively (the ledger stays repair's food).

report_refusal(read_id) — the v0.6 rung, still here

The behavioral retry over residual spans (Context intelligence) predates the chain and composes with it — it's part of the baseline the v0.7 numbers are measured against. Use it as the cheap first response to a refusal when the stack is armed; the chain is the deeper ladder behind it.

Ingest metadata — the attribution golden path, by default

New in v0.7 Chunk.meta closes v0.6's documented attribution limit. Pass meta= at ingest (recognized keys title / source / date; extras preserved) and it's captured onto the unit as source_meta, serialized round-trip, and rendered by the metadata-first default pool header — the measured [title | source | date] golden path without writing a pool_header callable:

retriever.add(
    "news:az-01",
    "Azure region refresh: three new EU regions come online in May...",
    meta={"title": "Azure region refresh", "source": "CloudWire", "date": "2026-05-02"},
)
# the default pool header now renders: [Azure region refresh | CloudWire | 2026-05-02]

Meta-less ingests are byte-identical to v0.6 (including their serialized form) — the unit-title fallback header still applies, and the pool_header callable still wins when you set one. constraints= matches against this same metadata, so one meta= line feeds both attribution and the repair ledger. MCP folder mode auto-wires it for watched files.

The knob-adoption ladder

Adopt one step at a time — each step is independently safe to stop at:

What the chain cannot break — the pinned guarantees

Each row is a pinned test in the library's suite, not a promise:

Observability

New events for on_event=...: gap_detector, constraints_matched, constraints_and_fallback, repair_applied ({unit_id, claims_added} per touched unit), reprobe, reprobe_skipped, serve_unserved, serve_unserved_skipped, pool_decomposed.

Next

• Integrating with your failure handling — the recipes: where the chain slots into the failure structure you already have.
• get() & Result — the full read signature and every Result field.
• Benchmark — the v0.7 composition numbers and their honest framing.