Repo: den-server (LSP + MCP frontends) · den/gen (projection lib + node.query facet) · nixd (upstream reference) · Date: 2026-07-20 · Status: approved scoping design (gate ladder + consumer analysis; vehicle = one den-server with LSP + MCP frontends over a shared query engine, built MCP-first; Gate-2 spike decides viability + routing, not the vehicle)
Scope the work and delivery order for extending Nix language tooling to understand
den's gen libraries, policy system, and native (gen-graph) graphs. This is a scoping
spec: it fixes the value ladder, the eval-model seam that governs it, the per-gate
capability delta over vanilla nixd and over vanilla NixOS, and the vehicle decision
point. It is not an implementation spec — each gate that survives graduates to its own
plan.md.
Two consumers are in scope — a human in an editor (LSP) and an agent via tools (MCP) — delivered as two thin frontends over one shared query engine (see One engine, two frontends). The gate ladder is defined in LSP terms first because that is the harder latency budget; the MCP is a cheaper, higher-value rendering of the same engine and reorders the delivery (see Consumer analysis and Delivery order).
Every decision below follows from how the server evaluates Nix. There are exactly two regimes, and the boundary between them is the architecture.
| E0 — external / cached-once | E1/E2 — live / resolved | |
|---|---|---|
| Evaluates | a fixed expr (den/gen schema), once | the working fleet — buffer + imports, re-eval on edit |
| Invalidation | never | on change (debounced), lazy, scoped |
| Answers | "what options/aspects/lib-APIs exist" | "what actually resolves for the entity under my cursor" |
| Cost | pay once, amortized | pays den's eval bill live (tens of seconds, single-thread-eval-bound) |
| Vehicle | vanilla nixd (config-only; Gate 1b adds one small idiom patch) | fork or separate den-lsp |
nixd is, by construction, an E0 machine: the option worker evaluates its provider expr
once into a cached nix::Value and never invalidates on edit
(nixd/include/nixd/Eval/AttrSetProvider.h:13-16). Both endgame asks — resolved policy
state, interactive graph navigation — are E2. The ladder therefore reads: extract
everything E0 gives for free, then cross the seam deliberately and once.
The load-bearing E2 primitive is position → resolved-node: map a cursor in the
libnixf AST to a node in the resolved den fleet / gen-graph. The inverse already exists —
den's unsafeGetAttrPos attribution (the den-rewrite migrator) — so resolved nodes carry
source positions and the map inverts. That primitive is what makes policy-resolution and
graph-nav engineering rather than research.
Confirmed by source read of the nixd tree, so Part I's "zero C++" claim is load-bearing, not hopeful:
- The option feature is generic, not NixOS-specific. It walks any attrset whose
leaves carry
_type == "option"(libnixt/lib/Value.cpp:46-48), readingtype.{name,description},description,declarationPositions(theunsafeGetAttrPosshape → goto),example/default(nixd/lib/Eval/AttrSetProvider.cpp:173-203). The NixOS default provider is itself just(lib.evalModules {...}).optionsfed through the same path (nixd/lib/Controller/LifeTime.cpp:31-37). - Submodule descent keys off
lib.typesconventions:type.name == "submodule"→getSubOptions,attrsOf→nestedTypes.elemType(libnixt/lib/Value.cpp:117-194). - A provider is added purely by config:
nixd.options.<name>.exprspawns anixd-attrset-evalworker holding a realnix::EvalState; completion/hover/goto iterate all option workers generically (Completion.cpp:345-353,Definition.cpp:269-274). No C++, no plugin — config only.
Two hard limits config cannot cross:
- Trigger context. Option completion fires only when the enclosing node is
NK_ExprAttrs(an attr binding). Variable/select chains (inputs', gen accessor pipelines) route through the idiom layer that is hardcoded to the literal namespkgs/lib(nixd/lib/Controller/AST.h:20-26). - E0 only. Everything is the declared schema; nothing resolved. There is one
NixOS-ism baked in C++ —
findAttrPathForOptionsstrips a leadingconfig.segment (nixd/lib/Controller/AST.cpp:340-342, marked FIXME) — a minor shim point for den's rooting.
The eval-model table above is one axis. Vehicle is a second, orthogonal axis — config-only vs C++ — and the two do not collapse into one. In particular there is a real "E0 but needs C++" quadrant that a naive "static ⇒ no C++" reading would miss:
| config-only | needs C++ | |
|---|---|---|
| E0 (static schema) | Gates 0–1 — option / aspect / lib projection | Gate 1b — gen-accessor idioms (var/select outside NK_ExprAttrs) |
| E2 (live resolved) | — (impossible: live eval needs the vehicle) | Gates 2–5 |
So there are two seams, not one:
- a minor seam at Gate 1b — the first C++, but still static E0. It only teaches nixd's
idiom layer (
AST.h:20-26, hardcoded topkgs/lib) to recognize gen accessors. Cheap, self-contained, and a plausible upstream contribution (make idiom vars config-driven), so it does not force the vehicle decision. - the major seam at Gate 2 — the E0→E2 crossing, where the fork-vs-den-lsp decision and the dominant risk (live-resolution latency) both live.
The gate ladder is grouped by these two seams.
Gate 0 — Options & aspect schema. A pure-Nix den.lib.lsp.optionsProjection that
emits the _type=="option" / lib.types shape the option worker walks. Two projections:
(a) den's module options; (b) the aspect registry modeled as an options tree — aspect
name = submodule, its settings = sub-options — so aspect-name completion, per-aspect
settings completion, hover (description), and goto (to the aspect's defining file via
declarationPositions) all fall out of the existing mechanism. Config:
nixd.options.den.expr, nixd.options.den-aspects.expr.
- Enables: complete den option paths + aspect names + aspect settings; hover (type + description); goto-def — anywhere written in attrs/binding position.
- Eval tier: E0 (den schema is stable library code; cache-once is correct).
- Constraint: the projection MUST be lazy — forcing
.optionsyields declarations without running the fx-pipeline/materialization. Declarations are static, so this is achievable and mandatory (den eval is expensive).
Gate 1 — gen library API surface. Extend the projection to the gen ecosystem: the 19
mkGenLibs keys, gen-schema registries, gen-graph constructors / labeledFrom,
gen-select accessors — each lib's API as an option-tree of callable members with
type/description/goto, with README / gen-specs/<lib>/REFERENCE.md theory citations
surfaced as hover text.
- Enables: complete/hover/goto/docs across the whole gen surface, in attrs position.
- Eval tier: E0. Limit: accessors used via var/select fall outside
NK_ExprAttrs(see limit #1) → handled by Gate 1b.
This is the ceiling of vanilla nixd (config-only).
Gate 1b — gen-accessor idioms. Teach the idiom layer to recognize den/gen accessor
roots (e.g. inputs', gen accessor pipelines) so completion/hover/goto work on var/select
chains, not only NK_ExprAttrs bindings (lifting limit #1). The chain resolves against the
cached gen schema value — still E0, no live eval. Implemented as a new idiom recognizer
beside pkgs/lib (nixd/lib/Controller/AST.cpp:170-270); best shape is a config-driven
idiom list (nixd.idioms) offered upstream, with a small fork patch as fallback.
- Enables: the Gate-1 gen-lib affordances extended to the idiomatic call sites where gen is actually written (accessor chains), not just attrs position.
- Eval tier: E0. Vehicle: first C++; upstreamable; does not force the Gate-2 vehicle decision.
- Risk: low — additive to a well-understood layer.
Gate 2 — Live fleet evaluator. The pivot. An eval backend that re-evaluates the
working fleet, invalidates on edit, holds a resolved den value, and answers position-keyed
queries. Ships no user-facing feature alone — it is the substrate Gates 3–4 stand on.
Implements the position → resolved-node map on top of den's unsafeGetAttrPos
attribution. This gate carries the single biggest risk in the spec: den live-resolution
latency vs the sub-second LSP interactivity budget — but note that wall gates the LSP
frontend only; the MCP frontend tolerates it (see Consumer analysis). It is also where
the vehicle decision is forced (below).
Gate 3 — Resolved policy state. Rides Gate 2 + den's STEP-5 resolution facet
(node.query / WS-QUERY / WS-RELATIONS): the LSP is a consumer of the resolution facet,
not a reinventor of policy dispatch. For the entity/aspect under cursor: which policies
fire (resolveArgsSatisfied guards), which provides.to-users / provides.to-hosts
reach this user/host, resolved aspect settings after default < env < host < policy
stratification. Surfaced as hover, code lenses ("3 policies apply here"), and goto from a
provides-consumer to the producing policy.
- Why it matters: the policy/effects system exists because resolution is non-local; the LSP makes non-local resolution visible at the edit site. This is the killer feature.
- Eval tier: E2. Risk: eval cost; correctness of guard introspection (the fire-once
{ self }vs stale entity-named-destructure subtlety).
Gate 4 — Interactive native graph navigation. Full custom protocol over gen-graph:
execute labeled queries (Brzozowski regex-over-labels, all 5 modes) from the editor;
neighbor navigation (goto edge target/source by label); path queries; rendering (code
lenses / DOT-or-webview / inlay hints showing resolved edges). New methods
dengraph/query|neighbors|resolvePath|render. Requires a gen-graph reflection API
(enumerate nodes/edges/labels + eval a query) and graph values carrying source positions.
- Enables: navigate the native graph as a first-class object — impossible as text in both nixd and NixOS. Eval tier: E2 + reflection. Risk: highest; most net-new surface; the strongest argument for a separate den-lsp (custom protocol + webview + code lenses beyond nixd's feature set).
Gate 5 (optional capstone) — den-native diagnostics & code actions. Surface den's own
laws/guards (den.derived field-guards a–g) as diagnostics; wire den-rewrite codemods
as LSP code actions (schema migrations as one-click quick-fixes).
- Eval tier: E2. Depends on: Gate 2's evaluator + Gate 3's resolution — not Gate 4; it is not sequenced behind interactive graph nav.
| Gate | vs vanilla nixd | vs vanilla NixOS |
|---|---|---|
| 0 | nixd completes NixOS options only; adds the entire den config surface (claims, provides, aspect settings) | NixOS has no tooling for a fleet config surface at all |
| 1 | nixd has zero concept of gen; adds complete/hover/goto/docs for 19 gen libs | n/a (no gen) |
| 1b | teaches nixd's pkgs/lib-only idiom layer to recognize gen accessors → complete/hover/goto on accessor chains, not just attrs bindings |
n/a (no gen) |
| 2 | nixd cannot evaluate your working config live, by design | nixos-rebuild evaluates but gives the editor nothing |
| 3 | categorically beyond nixd — it never resolves policy/provides | nixos-option shows a resolved value; den-lsp shows why — which policy fired, which guards passed, which provides-edges reached here |
| 4 | nixd/text cannot navigate a graph | NixOS module deps are implicit and invisible; den-lsp makes the explicit gen-graph queryable and visual |
| 5 | nixd has no domain diagnostics; den-rewrite migrations become quick-fixes | NixOS has no equivalent codemod-as-code-action story |
The gates above are framed for a human in an editor (LSP). But the highest-value features serve an agent at least as well, and an agent reasons differently enough to reorder the value ranking. Value splits by consumer:
| Capability | Human expert (editor / LSP) | Agent (tools / MCP) |
|---|---|---|
| Gate 0/1 — what the API is | low — you wrote den | high — agents hallucinate den/gen APIs |
| Gate 3 — what resolves | highest — can't hold the resolved fleet in your head | high — agents reason badly about non-local effects |
| Gate 4 — the graph | moderate (navigation) | value is impact queries, not clicking |
| completion (LSP headline) | useful while typing | near-worthless |
Two consequences drive the MCP design:
- An agent has no cursor. It has a question and a plan, and needs structured answers to semantic questions, not position-anchored UI. Completion — the LSP's marquee feature — barely helps; the agent needs its inverse: enumerate valid options, validate what it wrote, resolve what applies — not char-by-char completion.
- The Gate-2 latency wall mostly evaporates for an MCP. An LSP hover must return in ~200 ms; an agent tool call routinely takes 5–30 s and the agent just waits. So the expensive, high-value resolution features ship on the MCP without solving interactive latency — the MCP reaches Gate-3 value on a shorter path than the LSP.
Ranked agent (MCP) tools — each a thin wrapper over an engine query (see next section):
den.resolve(entity)→ structured resolved state (aspects, stratified settings, policies fired, provides reaching). Gate 3 as a query answer. Top capability; latency-tolerant, buildable before any latency hardening.den.check(files)→ structured pre-resolved diagnostics (den.derivedguards/laws with locations). Collapses the agent's edit →nix eval→ grep loop. (Gate 5, agent-facing.)den.schema/den.aspects.list/gen.lib.signature→ ground-truth enumeration. Kills API hallucination — the most common agent failure. Gate 0's projection re-exposed as tools (built once, serves both vehicles). Highest ROI.den.impact(change)/den.graph.query→ blast radius before a blind edit. Where the native graph earns its cost for an agent — programmatic impact, not interactive nav.den.migrate(rule)→ den-rewrite codemods invoked as a tool. Agents orchestrate deterministic transforms well and hand-edit across files badly.den.where(symbol)→ provenance viaunsafeGetAttrPos, robust against theid_hashre-key that greps miss.- (capstone)
den.dryRun(edit)→ resolved-diff of a proposed edit in a scratch overlay. Closes the agent loop without a rebuild; uniquely valuable to an agent.
Correctness caveat, sharper for MCP than LSP: resolve / check must be faithful
consumers of the real resolver (node.query), never a reimplementation — an agent will
confidently build on a subtly-wrong resolved state, so surface forcing failures explicitly
rather than returning a plausible-but-partial answer.
The LSP and MCP are not two projects. They are two thin adapters over one shared query engine, so building them separately would duplicate the expensive part.
┌─ LSP frontend ─┐ ┌─ MCP frontend ─┐ ← thin, divergent (protocol veneer)
│ position→subject│ │ tool args→subject│
└────────┬───────┘ └───────┬────────┘
└──────────┬─────────┘
┌───────────▼────────────┐
│ QUERY ENGINE (Gate 2) │ ← expensive, 100% shared
│ resolve/check/impact/ │ holds nix::EvalState, position↔node
│ where/enumerate │
└───────────┬────────────┘
┌───────────▼────────────┐
│ den-side reflection API │ ← pure Nix, 100% shared
│ node.query · projection │ the load-bearing investment
│ unsafeGetAttrPos · graph│
└─────────────────────────┘
- Layer 0 (den-side Nix API) and Layer 1 (query engine) are shared verbatim. Define
the engine API as protocol-neutral verbs —
resolve(subject),check(files),impact(change),where(symbol),enumerate(schema)— and both frontends ask the same questions. - Layer 2 (frontends) is the only divergent code, and it is thin. The LSP adapter has one extra front stage the MCP lacks — position → subject (cursor → entity, via libnixf); the MCP takes the subject from tool args. Additive, not conflicting.
Three tiers of "both", increasing in cost:
- Cheap tier (Gate 0/1) is trivially both — one pure-Nix projection is consumed by
vanilla nixd as
optionsconfig and by the MCP as enumeration tools. The projection is the shared artifact; no shared server required yet. - Engine tier (Gate 2–5) is both via one engine, two adapters. The MCP frontend is far cheaper to add than the LSP frontend (no position math, no incremental doc-sync, no sub-second budget — request/response JSON over stdio). Once the engine exists, the MCP frontend is nearly free.
- The only real cost of insisting on both: the LSP re-imposes the ~200 ms latency wall the MCP lets you dodge. "Both" costs the interactive-latency hardening (backgrounding / incremental eval / caching), not a second engine.
Vehicle, restated: one den-server (libnixf + libnixt as libraries) exposing the Layer-1
query API, with an LSP frontend and an MCP frontend over it. (nixd's nixd-attrset-eval
worker is already a query engine answering semantic queries over JSON-RPC on stdio, so a
fork could host both frontends against one worker; the clean separate den-server is the
better home for the resolved-state / graph endgame. Same pattern as rust-analyzer's
core-with-frontends.)
Gates 0–1 need no vehicle decision — they ship as a pure-Nix lib in den/gen against stock nixd. The decision bites only on crossing into E2:
- nixd fork. Reuse transport / AST / idioms / worker-spawn wholesale; add an eval tier that relaxes cache-once for a working-doc worker + new RPCs. Faster to first light; you carry a C++ fork and fight upstream's E0 assumption for every resolved/graph feature.
- Separate den-lsp. libnixf + libnixt as libraries, own the eval lifecycle and a den-native protocol (graph RPCs, code lenses, webview) from day one. More upfront plumbing (re-implement dispatch / doc-sync that nixd already has); no upstream friction; the honest endgame for full-interactive-graph + resolved-state + custom UI.
Recommendation. Given the Gate-3/Gate-4 choices (resolved state + full interactive graph + custom UI) and the MCP as the primary high-value vehicle (see One engine, two frontends), the endgame is one den-server with LSP + MCP frontends over a shared engine, built MCP-first. But do not commit the LSP-interactivity investment blind: spike the live evaluator as a throwaway nixd fork worker first. The spike answers the one question that governs the LSP frontend: can den resolve the entity-under-cursor inside an interactive (~200 ms) budget? — a question the MCP frontend does not have to pass, since it tolerates 5–30 s tool latency.
- If yes → build the den-server (MCP frontend first, LSP frontend second).
- If no → the MCP frontend still ships (latency-tolerant); the LSP interactivity
prerequisite becomes a den-side fast-resolution path (gen-resolve / a scoped
node.query), and only the LSP frontend waits on it.
Mitigations the spike must exercise: laziness (force only queried paths); scoped sub-fleet eval (not the whole fleet); background eval with stale-while-revalidate; a faster eval engine (e.g. Determinate) as a constant-factor floor. den eval is single-thread-eval-bound, so the interactivity budget is won by scoping the eval, not by parallelism.
No wait required, and building alongside is the better plan. The dependency is finer-grained than "den v2 must be finished":
- Cheap tier (Gate 0/1 + MCP enumeration) has zero den-hoag dependency. It reads the declaration surface (options, aspect registry, gen-lib signatures), which exists in den v1 today. Buildable immediately.
- The engine's resolution queries consume den-hoag's resolution facet (
node.query), which is already live. STEP-5 has shipped 4 of 6 sub-arcs — WS-QUERY, WS-RELATIONS, WS-DERIVED, and WS-ACL (the general relation-agnosticnode.queryprimitive +den.resolutionProducts). Materialization itself (materializeUnified) is on main. Soresolve/impact/whereconsume a shipped facet, not a future one. - The two remaining STEP-5 sub-arcs enrich specific answers; they do not gate the
architecture. WS-SETTINGS sharpens
resolve()'s stratified-settings view (thedefault<env<host<policycascade); WS-FN lands the forward.fn.<method>accessor (retiringdomainFor). Bind tonode.query(the stabilized capstone) and the.fn.accessor, and these land as enrichments while you build. - The LSP/MCP engine is itself a demanding witness of
node.query. WS-ACL framed the facet as "a general primitive with non-privileged witnesses"; this engine is a more demanding witness than ACL, so co-developing surfaces facet-generality gaps earlier than waiting would. That is an argument for overlap, not sequence.
Real cost of starting early — API churn against a moving v2 facet (WS-SETTINGS / WS-FN /
step-6 Compat still landing). Mitigation: bind the engine only to the stabilized general
primitive (node.query), consume forward accessors (.fn., not domainFor), and keep the
Layer-0 binding thin and swappable. The cheap tier targets v1's declaration surface today;
the resolved tier paces with the live v2 facet — which is 4/6 shipped, so "pacing with"
is not "waiting for."
Gate 0 ─→ Gate 1 (E0, config-only: den/gen pure-Nix projection lib)
│ └─→ MCP enumeration tools (schema / aspects / lib — same projection, ~free)
── minor seam: first C++ (LSP only) ──
Gate 1b (E0 + C++: gen-accessor idioms; upstreamable)
══ major seam: E0→E2 · build the shared QUERY ENGINE (Gate 2) ══
Gate 2 (engine)
├─→ MCP frontend FIRST (resolve / check / impact / migrate / where — latency-tolerant)
│ └─ delivers Gate 3 resolve · Gate 5 check+migrate · Gate 4-as-queries impact
└─→ LSP frontend SECOND (same engine + sub-second latency hardening; SPIKE gates this)
└─ delivers Gate 3 hover/lens · Gate 4 interactive nav · Gate 5 code actions
Sequencing notes: the Gate-0 projection is dual-consumed (nixd options config and MCP
enumeration) — one artifact. Gate 1b is LSP-only (accessor completion has no MCP analogue)
and independent of Gate 2. The MCP frontend precedes the LSP frontend: it reaches
resolved-state value without the latency hardening the Gate-2 spike gates. Gate 5 hangs off
Gate 3, not Gate 4.
- Gate-0 probe. Does gen-schema / neron
mkTypealready emitlib.types-shaped nodes (_type=="option",type.name,getSubOptions,nestedTypes.elemType), or does the projection need a type-mapping shim gen-schema-type → option-node? A cheap eval experiment; it is the entire risk of Part I. (Note: hola ownsevalModulesbyte-identically, so module.optionsare lib.types-shaped; the open question is gen-schema's own type layer, which gen-aspects builds on.) - Gate-2 spike. den live-resolution latency. Gates the LSP frontend's interactivity; the MCP frontend is latency-tolerant, so this determines how the resolved features render (and whether the LSP hover is interactive), not whether they exist. The MCP path ships regardless.
- Gate-4 reflection-API prerequisite. gen-graph must expose a stable reflection/query API (enumerate nodes/edges/labels + eval a labeled query) with graph values carrying source positions. A hard den/gen-side dependency — surfaced here to confirm early, mirroring Gate 3's reliance on the STEP-5 resolution facet; Gate 4 cannot start without it.
Each must be settled before its dependent gate is scheduled, per the "verify a spike's premise matches the real path" discipline — no gate rests on an unmeasured assumption.
- den options for the nixpkgs-module config surface as its own project — folded into Gate 0's projection, not separately specified.
- The lossy graph-as-tree interim (explicitly rejected by the owner in favor of Gate-4 full interactive nav; recorded here so the decision is not silently revisited).
- Formatting, rename, references, semantic tokens — inherited from nixd/libnixf as-is; no den-specific work.
Each surviving gate graduates to plans/YYYY-MM-DD-<gate>-plan.md. Immediate next actions
after this spec is accepted:
- Gate-0 probe (gen-schema type-node shape) — a day-scale eval experiment.
- On probe-pass, Gate-0 + Gate-1 plan (pure-Nix projection lib) — expose the same projection as MCP enumeration tools in the same plan (one artifact, two consumers).
- Gate-2 latency spike (throwaway nixd fork worker) — independent of Part I; gates the LSP frontend's sub-second budget, not the MCP frontend (latency-tolerant).
- Once the engine exists, build the MCP frontend first (
resolve/check/impact) — reaches resolved-state value without latency hardening — then the LSP frontend. - Gate-4 gen-graph reflection-API prerequisite — confirm shape + source-position carriage early (parallel to Part I; gates only Gate 4).