Implementation Roadmap
the single supervised path—from a calibrated uncertainty signal to a human-certified coupling, with the gates that decide everything

The framework argued that the object of design is the coupled system—human plus agent plus world—and that its signature mechanism is confidence-calibrated selective friction: smooth where stakes are low, deliberately resistant where being fluent is not the same as being right. This document turns that mechanism into something you can resource, build, and measure. It is deliberately not a menu of parallel workstreams. The mechanism's dependency graph is a chain, and the honest way to plan a chain is as one path with checkpoints, not four lanes that pretend to run abreast.

calibrated internal uncertainty signal → a harness integration point that exposes it to routing → a stakes × confidence friction router → certified only at the coupling level by human-subjects measurement.

Read left to right, three of those four pieces are serially downstream of the first. You cannot build the router's confidence axis before a usable signal exists; you cannot certify the coupling before the router exists to be certified. Presenting the work as four tracks (product, harness, research, specialist) invites four teams to sprint in parallel—and three of them would sprint straight into a wall, because their inputs are not ready. So this is one path. The word supervised is doing double duty, and both meanings are load-bearing: every stage carries a go/no-go gate so that scarce compute is never spent past a failure, and the terminal gate is literal human supervision—the coupling is certified only by measuring real overseers, which is the framework's decisive level and the one most AI organizations have no in-house competency to measure.

A single path is brittle if any link can stall the whole chain, so the path is built with two hedges that keep it moving. First, a stakes-only floor ships in parallel and immediately: friction keyed to stakes and reversibility depends on no unproven research, captures most of the safety value, and is buildable now. Second, the chokepoint signal has a today-buildable fallback—semantic entropy or self-consistency sampling—so the product is never blocked waiting on bespoke research. The committed path adds the confidence layer on top of the floor and gates it on the research actually validating.

Part 1 The Chokepoint, and Why Discrimination Is the Wrong Target

Every selective-friction system lives or dies on one quantity: a signal that says, for a given claim or action, how much the agent should be trusted right now. Everything downstream is plumbing and interface around that number. It is also the least-proven component in the chain, which is why it is the chokepoint—and why getting its success metric right matters more than any single experiment.

The instinct is to chase discrimination—the ability to separate true from false on average, summarized as AUROC (the area under the receiver-operating-characteristic curve; equivalently, the probability the signal ranks a random correct answer above a random wrong one). Discrimination is necessary but it is the wrong target. Friction routing needs calibration: when the signal says 0.2, the agent should be right about twenty percent of the time. A signal with excellent AUROC but poor calibration fires friction at the wrong moments—interrupting on confident-correct work, waving through confident-wrong work—and the predictable result is habituation: the user learns the pauses are noise and clicks past them, which is the exact failure selective friction exists to prevent (high). The load-bearing milestones are therefore, in order: a usable AUROC via the right objective; then expected calibration error and reliability diagrams; measured per claim, not per output; and holding under distribution shift, because the regime where the signal is needed is precisely the unfamiliar one.

There is news at the chokepoint, and it changes the resourcing. As of the latest runs in the energy-model line that this framework's research sits on, the chokepoint is no longer merely suspected—it has been probed directly. The original diagnosis was that the bottleneck is variance, not scale, fixable by an objective change costing one experiment on a frozen model rather than a multi-GPU training run. That experiment has now run, and the result is sharper than the diagnosis: the objective change did exactly what the math predicted to the variance, yet AUROC did not move—so the objective is not the bottleneck, and the open question narrows to backbone-versus-data. Part 3 lays out the diagnosis, the experiment, and the result in full, because the question it answers—"what has to change, and is it just scale and data?"—is the question the entire research arc has been circling.

Part 2 The Supervised Path

Here is the whole path on one page. The floor runs in parallel from day one; the numbered stages run in sequence, each behind a gate. Nothing past a failed gate gets resourced until the fork at that gate is taken.

The floor is the part of the framework that owes nothing to research and should not wait for it. It is: a per-domain stakes-and-reversibility taxonomy elicited from domain experts (a medical agent's notion of irreversible comes from clinicians, a legal agent's from lawyers—engineers cannot guess these); a router that maps stakes alone to a level of automation, from auto-execute through execute-then-notify, propose-and-confirm, suggest-options, down to no-assist, with the authority to change the level held by the human at all times; hard capability enforcement so the agent literally cannot call a tool outside its grant, plus two-phase (dry-run then commit) execution and undo windows for consequential actions; a side-effecting-action verification gate that blocks or downgrades rather than emitting-then-filtering; a lossless, machine-readable trajectory trace as a first-class artifact; and a first situation-awareness pilot to certify that the floor actually keeps a human meaningfully in the loop.

Stage 1 is the chokepoint: the objective fix, run on a frozen, already-trained backbone, costing a single re-run rather than a training campaign. Gate 1 has two parts because discrimination alone is insufficient: a clean prompt-level AUROC threshold and a per-claim calibration-error threshold after scaling, both holding under at least one held-out distribution shift. If Stage 1 passes, the bespoke signal proceeds to Stage 2. If it fails—if AUROC stays pinned near chance once the objective is fixed—the path does not stall: it forks to the semantic-entropy fallback and proceeds with that while backbone work continues separately. The router is never blocked on the research either way. This fork is the single most important sequencing rule in the document, and Part 3 explains why taking it cheaply, before any scale-up, is what keeps the program from paying multi-GPU prices to rediscover a ceiling.

Stage 2 wraps whatever signal cleared Gate 1, bespoke or fallback, behind a stable interface, so the rest of the system never depends on which method is underneath. Gate 2 is decision-theoretic, not statistical: the signal has to make routing better on held-out trajectories than routing on stakes alone, or it does not earn the added complexity. Stage 3 layers the confidence axis onto the floor router so friction now rises on stakes and on genuine internal uncertainty, and brings up the two surfaces that axis requires—a trajectory-legibility view that marks verified claims apart from inferred ones, and the lightweight calibration telemetry that captures overrides-with-reason and sampled awareness probes. Gate 3 is measured by the Stage 4 instrument and is the second of the two gates that decide everything: does calibrated friction actually improve the coupled human's error-catching and tighten their calibration, against both stakes-only and uniform-friction baselines, without taxing low-stakes throughput? Stage 4 is the certification study itself, which then becomes permanent in-product measurement.

Completeness is not the same as the critical path, and conflating them is how programs lose focus. The following are real and belong in the roadmap, but they are explicitly downstream or parallel, not gating: runtime intent-anchoring against goal drift; adversarial-robustness probing of the signal; a process reward model for agent action-traces; metacognitive / epistemic-uncertainty estimation; designed-in structured disagreement; fairness-parity measurement across affected populations including non-users; and institutional / multi-agent governance design. Part 5 treats each. They are sequenced after the chain clears, or run in slack capacity, because none of them is what the mechanism is blocked on.

Part 3 The Chokepoint, Resolved—Stage 1 in Full

This is the diagnosis and prescription for the energy-head signal in the Nested-Hope / energy-model line, which is where this framework's own research sits. The diagnosis is made with high confidence on mathematical grounds and remains pending the inexpensive experimental verification described below—the experiment is the gate, not the analysis. The line's clean, leakage-free result is a Phase-4 hallucination-self-detection AUROC of 0.586 on a 220M-parameter model (Variant A), up from 0.516 on the prior 181M model (v10) under identical methodology—a real but small +0.070 gain. Variant B, which adds a Self-Modifying Titans memory, is designed but untrained, because a full run needs multi-GPU compute. The question is exact: what needs to change, for both variants A (CMS) and B (Titans), to get a high AUROC—and is it just scale and data? The short answer is the one most people do not want to hear, so it goes first.

AUROC between two roughly-Gaussian score distributions is a single closed-form quantity—the standardized separation between them: Φ( Δμ / √(σ_c² + σ_h²) ), where Δμ is the gap between the mean energy of correct and hallucinated completions, the two sigmas are the within-class spreads, and Φ is the normal cumulative distribution. The crucial fact: AUROC depends on the gap divided by the pooled spread. A signal can grow the numerator without limit and barely move AUROC, if the denominator grows in step. That is exactly what the logged training data show.

Invert those two AUROCs and the mechanism is laid bare. A 0.516 AUROC is a standardized separation of Φ⁻¹(0.516) ≈ 0.04; a 0.588 AUROC is ≈ 0.22. Pair each with its logged gap and the pooled standard deviation is implied: it grew from about 0.047/0.04 ≈ 1.18 to about 0.410/0.22 ≈ 1.85 as training pushed the gap up. The variance ate the gap. An independent run in the same line logged the score distributions directly—correct completions at mean 1.586, spread 0.594; hallucinated at mean 1.774, spread 0.532—and its pairwise (Wilcoxon) AUROC came out to 0.594, matching the Gaussian-overlap prediction to three decimals. The distributions are not failing to separate their means; they are failing to separate relative to how wide they are.

This indicts the objective as the prime suspect. The contrastive / InfoNCE / margin family that the energy head was trained under maximizes mean separation—the numerator—and is structurally blind to within-class spread—the denominator. So it can grow the gap epoch after epoch while AUROC sits stuck in the 0.52–0.59 band across every lever that was tried: more epochs, larger margin, different data, different learning rate. On this analysis the wrong quantity was being optimized—a hypothesis the decisive experiment below was built to test directly. To reach AUROC 0.90 the standardized separation must hit Φ⁻¹(0.90) ≈ 1.28—between five and thirty times the separation these runs achieve, depending which run you anchor to. No amount of growing the gap alone gets there if the spread grows alongside it.

One nuance keeps the diagnosis honest. AUROC is invariant to any monotone rescaling of the score, so “shrink the variance” is not itself the lever—you cannot improve a fixed head's ranking by squashing its spread. What the corrected objective does is re-learn the head's weights to order pairs correctly, and a tighter within-class spread is the symptom of success, not the mechanism. The interpretive consequence is what makes the experiment decisive in both directions: if a head trained to optimize the rank statistic directly still cannot rank held-out pairs, the failure is no longer attributable to the loss—the factuality signal is, to the resolution of a small probe, not linearly present in the frozen representations at all. A null result under the corrected objective is evidence about the backbone, which is precisely the question the fork exists to answer (high).

The verdict, stated plainly: not just scale, and not just data. The v10→A gain of +0.070 was real and came from a stronger backbone (validation loss 3.77→3.07, roughly a 49% perplexity reduction), so representation quality clearly contributes. But linear extrapolation of that trend puts the perplexity reduction needed to reach 0.90-by-backbone-alone far out of practical reach, and scale does nothing for a variance problem while the objective that shapes the score distribution is unaddressed. The ordering of leverage is therefore: objective first, representation second, scale a distant third—and "objective first" is now literal history, not just advice: it was the cheapest thing to test, it has been tested, and ruling it out (the update above) is what legitimately promotes representation-versus-data to the live question. Anyone who reaches for a bigger model or more data before settling the loss is spending the most money on the least-leveraged variable.

The whole question resolves with one inexpensive, falsifiable experiment on a single GPU, with the Variant-A backbone frozen at its validation-3.07 weights so nothing about language modeling is in play. Replace the energy-head loss with a differentiable surrogate for AUROC itself—the smooth Wilcoxon–Mann–Whitney statistic, mean over cross-pairs of σ(E_halluc − E_correct), which is the soft form of the rank statistic AUROC measures—and add an explicit within-class variance penalty, λ·(σ_c² + σ_h²). Equivalently, maximize Fisher's discriminant ratio, (μ_h − μ_c)² / (σ_c² + σ_h²), which targets the standardized separation directly. The two terms are complementary, not redundant: the WMW surrogate's gradient saturates once a pair is well-separated, so the Fisher / variance term keeps pressure on the tails where the overlap actually lives. Re-run the Phase-4 calibration and read the AUROC.

Four protocol details turn that sketch into an experiment whose number can be trusted. First, the contrast is the experiment. Train the head twice from an identical fresh initialization on an identical prompt-level train/validation/holdout split—once under the legacy margin loss, once under the WMW-plus-variance loss—and report both held-out AUROCs side by side. A single WMW number is unattributable; the margin baseline on the same frozen features is what isolates the objective's effect from everything else. Second, cache the features. A frozen backbone's pooled head inputs are constants, so extract them once (inference mode, one forward per sequence) and train the head on cached vectors: the experiment then costs minutes of head training after one extraction pass, large batches become free—which matters, because the WMW statistic is computed over B×B cross-pairs and its gradient quality scales with batch—and it removes a quiet mismatch where the head previously trained against dropout-noised backbone states but was evaluated against clean ones. Third, the sigmoid temperature must sit on the order of the within-class energy spread (roughly 0.1–0.5 for this head). Set τ much larger than the spread and the soft rank goes nearly flat in exactly the regime that matters: by symmetry it degenerates toward a function of the mean gap alone—reproducing the very blindness being fixed. This was verified empirically on score distributions matching the logged runs: at τ = 1.0 the surrogate barely moves while true AUROC swings from 0.60 to 0.96; at τ ≈ 0.25 it tracks. The variance term is the hedge here too—its gradient is alive regardless of τ. Fourth, early-stop on the validation split and touch the holdout exactly once per objective, so the reported figure is neither an arbitrary-epoch accident nor contaminated by model selection.

The fork is only as valid as its data, and there is one way to invalidate it that looks like diligence: running it on constructed pairs. A file of hand-built or template-generated correct/wrong answers—atomic strings like 4 versus 5, Lima versus Cusco—tests whether a small head can separate two strings somebody else labeled, which is a different task from detecting the model's own errors and a far easier one. A head can pass it by exploiting whatever regularities the construction process left behind, and the result is the most expensive error available to this program: a false pass of Gate 1 that green-lights the Variant-B spend on a number that will not replicate on real generations. The decisive experiment runs on the model's own harvested generations—the same requirement change (4) below states for training, applied to measurement. Constructed pairs are legitimate only as a smoke test of the machinery, never as the verdict. A standing diagnostic belongs next to the AUROC wherever it is reported: the correlation between energy and raw completion length on the holdout. A high correlation does not prove the head is reading surface form, but a strong AUROC riding on one should be treated as unproven (high).

The number also comes with an error bar that the resourcing should respect. By the standard Hanley–McNeil approximation, a measured AUROC of 0.75 on a prompt-level holdout of 40 prompts (one correct and one hallucinated completion each) carries a 95% interval of roughly ±0.11; at 80 holdout prompts, ±0.08; at 150, ±0.055. (Same-prompt completions correlate, so these intervals are if anything slightly optimistic.) The consequence is a two-round design that matches the gate structure. A harvest of roughly 200 triples—a 40-prompt holdout under an 80/20 split—is enough to answer the fork, because a true jump from 0.59 clears the interval. It is not enough to certify the ≥0.75 threshold with any precision. Certification within about ±0.055 wants a holdout near 150 prompts, roughly 750 triples under the same split—a larger harvest that is only worth running after the cheap fork comes back green. Fork first on the small harvest; certify Gate 1 on the large one (high).

All four changes are cheap and live in the calibration phases on the frozen backbone—the AUROC fix is not a pretraining change. (1) Objective. The WMW/AUROC surrogate plus variance penalty, replacing pure margin/InfoNCE. This is the load-bearing change and the one the decisive experiment tests. (2) Readout source—corrected. An earlier draft of this change said to tap the slow memory levels as “bands integrating over 128- and 512-step windows.” That description does not match the architecture as built, and the correction matters. In Variant A the CMS level frequencies govern gradient-update cadence during training—how often a level's accumulated gradients are applied—not activation-time integration; every level is a stateless position-wise transform that fires on every forward pass, and nothing in Variant A maintains a 128- or 512-token activation window at inference. The honest Variant-A version of this change is therefore a readout over representational depth, not time: pool the energy head's input from the last few blocks' residual streams as well as the final state, with the head's weights shared across bands so the parameter count is unchanged and the comparison against the final-only head is clean. The genuinely temporal version of the hypothesis—that factual coherence lives in a long-range integrator—only becomes testable in Variant B, whose Self-Modifying Titans memory state is the one component in the line that actually accumulates across the sequence; there, the readout target is the memory state itself. The sharp caveat survives the correction: the slow-cadence levels are documented to specialize on long-range format structure—the model's entire format-collapse history is the slow levels over-weighting positional formatting—so any expanded readout risks scoring format-coherence rather than factual-coherence, and the ablation must explicitly check that the readout tracks factuality and not surface form (the energy–length correlation above is the cheap first probe). Confirm or kill; do not assume. And because this change is a secondary hypothesis with a live confound, it stays out of the decisive experiment: run the objective contrast on the unchanged final-state readout first, and ablate the depth readout separately on the cached features, where it costs minutes. (3) Positive volume. The harvest yields only on the order of 3–29 correct generations per few-hundred prompts, which both starves the head of diverse positives and is the direct cause of the Phase-6 leakage discussed below. Raise correct-pool volume—a larger best-of-N, or a cheaper correct-answer verifier—and split at the prompt level before any combinatorial pair expansion. (4) Task alignment, correctly diagnosed. The head's high historical numbers leaned on cached external human-preference pairs (UltraFeedback and HH-RLHF, 5,000 of them) as the contrastive volume, because the self-generated harvest was too sparse to train on. That is a task mismatch—ranking other models' outputs by human preference is a different problem from detecting one's own hallucinations, and its quality signal lives in features a 220M backbone never encoded—but it was a crutch forced by sparsity, not a free choice. The fix is therefore coupled to change (3): make self-generated correct/hallucinated pairs abundant enough that the head can train on the actual task, rather than treating the preference data as a villain to be removed while leaving the head with nothing to learn from. Self-detection is, definitionally, detection of the model's own errors.

The single most important instruction about Variant B is a sequencing one: do not train it until the Variant-A objective experiment has run. If the variance diagnosis holds, a Self-Modifying Titans memory buys nothing on AUROC—better long-context recall is simply not the same capability as factuality discrimination—and a full B run would pay multi-GPU prices to rediscover the same 0.59 ceiling with a more expensive model. This is the concrete reason the path is sequenced and gated rather than parallelized: the gate at Stage 1 exists precisely to stop a premature, costly B run.

When B is eventually trained, two things govern it. First, scale is a complement to the objective fix, not a substitute—expect it to add a few points on top of a fixed head, consistent with the roughly +0.070-per-backbone-generation trend, not to rescue a broken objective. Second, and more subtle: a self-modifying memory means the energy head's input distribution drifts at inference time, because the memory the head reads from is being rewritten as the model runs. The head must therefore be trained against post-update representations, and its calibration must be re-verified under that drift. Skip this and B is plausibly worse-calibrated than A despite better recall—a regression that an AUROC-only evaluation would miss entirely and a calibration-aware one would catch. This is the architecture-specific reason calibration, not discrimination, is the gating metric. B is also where the temporal-readout hypothesis from change (2) gets its first legitimate test: the Titans memory state is the line's only true cross-sequence integrator, so reading the energy score from it—rather than from the CMS levels, which integrate nothing at inference—is the experiment that the “factual coherence is long-range” claim has actually been waiting for, and the same drift requirement applies to that readout doubly, since the state being read is the very thing rewriting itself.

Raising AUROC and making the signal usable for friction are different jobs. Once discrimination is acceptable, map energy to a calibrated probability with temperature scaling or isotonic regression fit on a held-out set, and validate with expected calibration error and reliability diagrams—per claim, and holding under distribution shift. Calibration does not raise AUROC; it makes a given AUROC trustworthy as a routing input. The provisional Gate-1 thresholds—clean prompt-level AUROC at least about 0.75 and per-claim ECE at most about 0.1 after scaling, surviving at least one held-out shift—are starting points, to be set properly by the cost asymmetry between false friction and missed errors, which is a joint product, engineering, and human-factors decision, not a number to pull from the air.

Part 4 Certifying the Coupling—Stage 4 in Full

This is the framework's decisive level and the one that needs a competency most AI organizations lack: experimental human-factors methodology, not ML benchmarking and not product analytics. What follows is the protocol skeleton an experimental psychologist would start from for an oversight / supervisory situation-awareness study—rigorous enough to be the certification gate, explicit about the gaps a specialist must close to bring it to full fidelity. It is a starting point, not a finished design, and the closing subsection says exactly where it is unfinished.

Four constructs carry the study, each with an operational definition that makes it measurable rather than rhetorical. Situation awareness is operationalized through Endsley's three-level model—Level 1, perception (does the supervising human notice what the agent did and on what evidence); Level 2, comprehension (do they understand what it means for the task); Level 3, projection (can they anticipate where the agent's trajectory is heading). Appropriate reliance—Lee and See's framing, the canonical reference for designing trust to match actual reliability—is decomposed with signal-detection theory: on trials carrying a deliberately seeded agent error, a correct override is a hit and a missed error is a miss; on correct-agent trials, an unnecessary override is a false alarm. The summary measure is sensitivity, d′, over the seeded errors, together with the response bias, which is what separates genuine error-catching from indiscriminate distrust—a user who overrides everything has no hits worth crediting. Trust calibration is the correspondence between the human's trust and the system's actual reliability, kept distinct from raw trust level (high trust in a reliable system and high trust in an unreliable one are opposite findings). Takeover quality is the latency and the accuracy of human resumption after the agent hands control back—the operational residue of the out-of-the-loop problem the framework inherits from automation research.

Situation awareness is measured with two complementary instruments because each has a failure mode the other covers. SAGAT (the Situation Awareness Global Assessment Technique) freezes the task at unpredictable points, blanks the displays, and queries the operator about the current situation across all three SA levels, scoring answers against the logged ground truth; it is the gold standard but it is intrusive and cannot run in production. SPAM (the Situation Present Assessment Method, from Durso and colleagues 1998) probes in real time without freezing the task and scores on response latency, trading some measurement purity for far lower intrusiveness, which makes it the instrument that can survive in-product. Trust is tracked with a validated trust-in-automation scale (Jian, Bisantz, and Drury 2000) administered repeatedly to capture the trust trajectory, not a single snapshot. Override-with-reason and takeover telemetry come from the in-product calibration instrumentation the path builds at Stage 3.

A factorial design crosses the three independent variables that the framework's claims actually concern: level of automation (suggest / execute-on-approval / execute-then-inform) × friction regime (none / uniform / selective-stakes-calibrated / selective-confidence-calibrated) × stakes (low / high, with reversibility crossed in). The design is mixed: stakes and error-presence vary within subject, while the friction regime varies between subjects, because trust formed under one friction regime would contaminate judgments under another if the same person saw several—trust does not reset between blocks. Task order is counterbalanced, and agent errors are seeded at a fixed base rate unknown to participants, so that error-catching is measurable without inducing the blanket suspicion that would itself destroy the reliance measure.

The tasks are stakes-graded agentic tasks within a single bounded domain—bounded so the domain stakes taxonomy from the floor applies cleanly—each with a fully logged ground-truth trajectory, and a controlled fraction carrying a seeded agent error of the kinds the framework names: a wrong tool call, a misread value, a plausible-but-wrong drift toward a nearby easier goal. The seeded error is the methodological linchpin. It is what converts the vague question "did the human stay engaged" into a measurable hit-or-miss on a known fault, and without it the central dependent measure—d′ on errors—cannot be computed at all.

The raw measures are the SAGAT and SPAM scores per SA level, the trust-scale trajectory, and the override and takeover telemetry. From these the framework's four coupling metrics are derived. Calibration is the correspondence between the human's stated confidence and the joint system's actual correctness—a reliability diagram plotting the human's confidence against whether the human-plus-agent outcome was right—which is the framework's overconfidence-decoupling measure made concrete. Appropriate reliance is the d′ and bias on seeded errors. Retained situation awareness is the SAGAT comprehension and projection scores, the Level-2 and Level-3 numbers being the ones that degrade first under complacency. Takeover quality is the latency plus post-takeover accuracy. The decisive contrast the whole study exists to resolve is precise: does selective confidence-calibrated friction raise d′ and tighten calibration without degrading low-stakes throughput, measured against uniform friction (predicted to habituate and to flatten d′) and against stakes-only friction (the floor)? That contrast is Gate 3, and it is the second of the two gates that decide everything.

The skeleton above is defensible but unfinished, and naming the gaps is part of the rigor rather than an admission against it. A specialist closes them: a power analysis fixing the sample size from the smallest effect worth detecting—the friction-regime contrast is the primary endpoint, and the seeded-error base rate is what drives statistical power on d′, so the two must be set together. Construct-validity work on appropriate reliance, because the signal-detection operationalization, though defensible, is contested, and the bias-and-threshold modeling needs care to avoid crediting indiscriminate behavior. Ecological validity, because a lab under-induces the real cost of being wrong that drives real oversight, so a field or in-product arm using the SPAM-style probes is needed to triangulate against the controlled lab arm. Resolution of the SAGAT-freeze intrusiveness against SPAM-latency-noise tradeoff, per deployment. Learning and order effects across stakes conditions, and individual differences—domain expertise, dispositional propensity to trust automation—entered as covariates rather than ignored. Pre-registration of the friction-regime hypotheses, and IRB / ethics review—with the specific note that seeded errors at a base rate unknown to participants constitute an incomplete-disclosure design, which carries a debriefing obligation and a justification burden (the deception is methodologically necessary: disclosure of the base rate would itself induce the blanket suspicion that destroys the reliance measure) that the protocol must state rather than leave for the board to discover.

Part 5 The Rest of the Path

The remaining components are organized here by their position relative to the chain—what feeds the floor, what the calibrated-friction stage needs, and what is genuinely downstream—rather than as four parallel tracks, because the parallel framing is the thing this roadmap exists to correct.

The harness supplies the classical control structure the raw model lacks, and it is where constraint-first is actually compiled in. The verification gate / constraint compiler runs checks before any side-effecting action or asserted claim and blocks or downgrades rather than emitting-then-filtering—and here is where the framework slightly outruns current capability, stated honestly: the side-effect half is genuinely constraint-first and shippable today (block any tool call outside the capability grant), while the factual-assertion half mostly is not, because no current technique can guarantee at the architecture level that a language model will not assert an unverified claim. The available assertion tools—extract atomic claims, bind each to a retrieved span via an entailment check above a threshold, reject novel entities in consequential slots—are check-after-generate, the very guardrail-after pattern the framework dislikes; constrained decoding remains immature. The honest framing is that true constraint-first is here for actions and only partial for assertions. The structured trajectory trace is a lossless, machine-readable per-step record—state, goal stack, tool, arguments, result, claims asserted and their types, confidence, evidence pointers—and its non-obvious requirement is capturing the goal/intent state most agent frameworks omit, without which intent drift cannot later be detected; deterministic replay from a trace is the debugging and offline-eval substrate. Capability enforcement and reversibility is the lowest-research-risk item in the program: idempotency keys, two-phase commit, soft-delete with undo windows, compensating transactions for multi-step rollback. Runtime intent-anchoring against drift is off the critical path but deep: maintain an immutable intent spec and score each proposed action against it, halting and re-confirming when drift exceeds threshold—with the load-bearing caveat that the anchor must be external (the immutable spec and retrieval ground truth), never the model criticizing itself, because reflection-style self-critique can amplify initial errors rather than converge on truth.

The design thesis inverts the historical objective—place friction, do not remove it—and designs against the system's own fluency, because fluency breeds the over-trust the framework is trying to defeat. The friction router is the operational heart and the home of the framework's disclosure pattern; it ships on stakes alone at the floor and gains the confidence axis at Stage 3, and it can start as an explicit, auditable rule table before any learned routing. The trajectory-legibility surface shows what was done, why, on what evidence, with what confidence, and—the sharp part—marks verified claims apart from inferred ones at the point of assertion, which means deliberately resisting the instinct to smooth the trace into confident narrative: inferred and low-confidence steps must read as more tentative, not equally polished, because a beautiful fluent trace is precisely what breeds misplaced trust (this is a genuine open UX question, not a settled pattern). Calibration instrumentation as a product surface collects the coupling metrics in-product—override-with-reason, acceptance, sampled awareness probes—under a hard constraint: every probe is itself friction, and uniform friction is self-defeating, so probes must be sparse, varied in timing, and stakes-weighted. Designed-in structured disagreement is off the critical path: above a stakes threshold, surface the strongest case against the recommended action as a first-class element rather than a mode someone has to remember to invoke—and measure whether it ever changes a decision, because if it never moves one it is theater and should be cut.

Three research programs sit downstream of the signal fix. Adversarial robustness of the internal signal applies the framework's own skepticism about observable process—its citation of alignment-faking work—to the signal itself: is it causal and robust, or a surface correlate that degrades under prompt framing, jailbreaks, or distribution shift, or that optimization pressure can make confident-looking while wrong? Causal interventions—activation patching and ablation—distinguish a causal signal from a correlate, and this probe is necessary before the signal gates anything consequential. A process reward model for agent action-traces generalizes step-level process supervision from reasoning chains to agent actions—was this tool call appropriate, did this step preserve intent, was this action reversible when it should have been—with two hard parts: defining the step-level reward (what makes an action good is less crisp than whether a math step is correct) and the labeling cost, plus the standing risk of reward-hacking the process reward itself. Metacognitive / epistemic uncertainty is the self-aware-agent target: a signal that distinguishes "I lack the information" from "I am confidently asserting," and epistemic uncertainty (a missing fact, reducible by retrieval) from aleatoric uncertainty (irreducible, like a coin flip), because the two should trigger different friction—abstention and clarification versus verification. Semantic entropy is the buildable-today baseline here and doubles as the chokepoint's fallback signal: resample the answer and measure entropy across semantically distinct meanings; high entropy across distinct meanings indicates epistemic uncertainty.

Three items are subject-matter expertise rather than ML or product. Domain-specialist elicitation is on the floor, not off it, because the friction router and the verification gates both need domain-specific definitions of high-stakes, irreversible, what counts as evidence, and which entities matter—elicited from clinicians, lawyers, or compliance officers, not guessed by engineers. Fairness-parity across affected populations operationalizes the framework's least-developed pillar: measure disparate impact in the coupled system, not just the model—including non-users who live with the outputs, and including the possibility that friction and automation-level policies are themselves applied unequally—while acknowledging that fairness criteria provably conflict and the choice among them is value-laden, not deferrable to math. Institutional / multi-agent governance is the most speculative item and is sequenced last and scoped as research, not delivery: persistent role-and-norm templates and constitutional structures in which independently-incentivized AI systems check one another, on the argument that dyadic preference-tuning cannot govern societies of agents.

The Two Gates That Decide Everything

If only two numbers from this entire roadmap were tracked, these are the two. Gate 1: does the energy head clear roughly 0.75 clean prompt-level AUROC, calibrated to about 0.1 ECE per claim under shift, once the objective is fixed—and if it does not, the signal-first path forks to the semantic-entropy fallback rather than stalling. Gate 3: does calibrated friction actually improve the coupled human's reliance and calibration, measured by someone qualified to measure it, against both stakes-only and uniform-friction baselines, without taxing low-stakes work. Gate 1 decides whether the bespoke research is worth its compute. Gate 3 decides whether the entire mechanism delivers on its premise. Everything else in this document is downstream of those two, and resourcing should follow that fact rather than the more comfortable fiction of four tracks advancing in parallel.

Key Takeaways

1. The mechanism's dependency graph is a chain, not four lanes: signal → integration → router → human certification. Three of the four pieces are serially downstream of the first, so the work is one supervised path, with go/no-go gates, not four parallel tracks.
2. For the chokepoint signal, calibration is the success metric, not discrimination. High AUROC with poor calibration fires friction at the wrong moments and trains the habituation that selective friction exists to prevent.
3. The chokepoint was first diagnosed as variance, not scale: the energy gap grew ~8.7× in the logs while AUROC barely moved, because the contrastive objective maximizes mean separation and is blind to within-class spread. That made the objective the prime suspect—a hypothesis cheap enough to test directly, which is what Takeaways 4–5 report.
4. Is it just scale and data? No. Ordering of leverage is objective first, representation second, scale a distant third. The objective fix is a differentiable AUROC surrogate (Wilcoxon–Mann–Whitney) plus a variance penalty, testable on a frozen backbone for the cost of one re-run—with the surrogate's temperature set on the order of the within-class energy spread, because a too-soft sigmoid degenerates back into mean-gap sensitivity, the very blindness being fixed.
5. That one experiment is a clean fork — and it has now run: both objectives landed at chance, so the objective is definitively not the bottleneck (a relative result, immune to how the eval was built). But it ran on constructed pairs, so whether the residual gap is the backbone or the data is still open; the same contrast on the model's own harvested generations, at a stronger and more factually-trained backbone, is what distinguishes a representational ceiling from a data-construction one. Do not pay for a Variant B scale-up before that harvested re-run.
6. The fork is only as valid as its data and its error bars. Run it on the model's own harvested generations—constructed pairs test string-separation, not self-detection, and can fake a pass—with the legacy objective as a controlled baseline on the same split, and read the AUROC with its interval: a 40-prompt holdout carries roughly ±0.11, enough to answer the fork but not to certify the 0.75 threshold, which wants a holdout near 150 prompts.
7. A self-modifying memory drifts the energy head's input distribution at inference time, so the head must be trained on post-update representations and re-calibrated under that drift—a regression an AUROC-only evaluation would miss. This is the architecture-specific reason calibration is the gate.
8. A stakes-only floor ships now, in parallel, with no research dependency, and captures most of the safety value. The confidence layer is a gated second increment, with semantic entropy as a today-buildable fallback so the path is never blocked on bespoke research.
9. The decisive level—the coupling—is certified only by human-subjects situation-awareness measurement, a competency most AI organizations lack. Seeded agent errors and a signal-detection (d′) reading of reliance are the methodological linchpins; the friction-regime contrast is the primary endpoint and Gate 3.

References

Methodological references for this roadmap. The framework's full bibliography is at https://cognitive.link

1. Bainbridge, 1983. Bainbridge, L. (1983). Ironies of automation. Automatica, 19(6), 775-779. https://doi.org/10.1016/0005-1098(83)90046-8
2. Behrouz et al., 2024. Behrouz, A., Zhong, P., & Mirrokni, V. (2024). Titans: Learning to memorize at test time. arXiv. https://arxiv.org/abs/2501.00663 (the Nested Learning / Continuum Memory System / Self-Modifying Titans primitives referenced in Part 3 originate in this line of work.)
3. Durso et al., 1998. Durso, F. T., Hackworth, C. A., Truitt, T. R., Crutchfield, J., Nikolic, D., & Manning, C. A. (1998). Situation awareness as a predictor of performance in en route air traffic controllers. Air Traffic Control Quarterly, 6(1), 1-20. Link. (origin of the SPAM real-time probe method.)
4. Endsley, 1995a. Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32-64. https://doi.org/10.1518/001872095779049543
5. Endsley, 1995b. Endsley, M. R. (1995). Measurement of situation awareness in dynamic systems. Human Factors, 37(1), 65-84. https://doi.org/10.1518/001872095779049499 (the SAGAT freeze-probe technique.)
6. Endsley & Kiris, 1995. Endsley, M. R., & Kiris, E. O. (1995). The out-of-the-loop performance problem and level of control in automation. Human Factors, 37(2), 381-394. https://doi.org/10.1518/001872095779064555
7. Farquhar et al., 2024. Farquhar, S., Kossen, J., Kuhn, L., & Gal, Y. (2024). Detecting hallucinations in large language models using semantic entropy. Nature, 630, 625-630. https://doi.org/10.1038/s41586-024-07421-0
8. Fisher, 1936. Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(2), 179-188. https://doi.org/10.1111/j.1469-1809.1936.tb02137.x (the linear discriminant ratio used in Part 3.)
9. Greenblatt et al., 2024. Greenblatt, R., Shlegeris, B., Sachan, K., & Roger, F. (2024). Alignment faking in large language models. arXiv. https://arxiv.org/abs/2412.14093
10. Guo et al., 2017. Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. Proceedings of the 34th International Conference on Machine Learning (ICML), 1321-1330. https://arxiv.org/abs/1706.04599
11. Hanley & McNeil, 1982. Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29-36. (the AUROC standard-error approximation behind the holdout-size arithmetic in Part 3.)
12. Jian, Bisantz, & Drury, 2000. Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000). Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics, 4(1), 53-71. https://doi.org/10.1207/S15327566IJCE0401_04
13. Kuhn et al., 2023. Kuhn, L., Gal, Y., & Farquhar, S. (2023). Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. International Conference on Learning Representations (ICLR). https://arxiv.org/abs/2302.09664
14. Lee & See, 2004. Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50-80. https://doi.org/10.1518/hfes.46.1.50_30392
15. Lightman et al., 2023. Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., & Cobbe, K. (2023). Let's verify step by step. arXiv. https://arxiv.org/abs/2305.20050
16. Parasuraman & Manzey, 2010. Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52(3), 381-410. https://doi.org/10.1177/0018720810376055
17. Parasuraman, Sheridan, & Wickens, 2000. Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A, 30(3), 286-297. https://doi.org/10.1109/3468.844354
18. Wang et al., 2023. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2023). Self-consistency improves chain of thought reasoning in language models. International Conference on Learning Representations (ICLR). https://arxiv.org/abs/2203.11171
19. Yan et al., 2003. Yan, L., Dodier, R., Mozer, M. C., & Wolniewicz, R. (2003). Optimizing classifier performance via an approximation to the Wilcoxon-Mann-Whitney statistic. Proceedings of the 20th International Conference on Machine Learning (ICML), 848-855. https://cdn.aaai.org/ICML/2003/ICML03-110.pdf (the differentiable AUROC surrogate used in Part 3.)