[Story]: Chunks carry no link to their graph node: hosts must reconstruct the join by string-building

[stephane-segning]

Summary

Chunk carries no reference to the GraphNode it is the body of. A host that consumes both halves of
a walk — the crate's stated purpose, "semantic chunks (ready to embed for search) and a structural call
graph … the caller decides where the output goes" — has to reconstruct the join itself, by string-
building {file_path}#{start_line + 1}:{symbol_name} and hoping it matches a node_id.

That reconstruction is an undocumented, untested coupling between two passes that were never
specified to agree, and it requires the caller to know two conventions the README documents
separately and never connects: chunk lines are 0-based, graph lines are 1-based, and a callable's
graph label gets a () suffix while its node id does not.

Proposal: emit the link where the information already exists — inside the walk.

Why this matters for a consumer

The join is what makes the two outputs worth more than the sum of their parts. With it, one query
answers "what code is semantically relevant to this question, and what calls it" — retrieve by
embedding, then traverse calls from the node the retrieved chunk belongs to. Without it, a host
either ships a fragile string-matching heuristic or treats the two outputs as unrelated datasets.

Current behaviour

pub struct Chunk {
    pub file_path: String,
    pub language: String,
    pub chunk_type: String,
    pub symbol_name: Option<String>,
    pub start_line: i32,   // 0-based
    pub end_line: i32,
    pub content: String,
}

WalkOutput { chunks, graph, stats } returns both, related only by convention.

How well does reconstruction actually work?

Measured rather than assumed, against main @ 293d12f with build_graph(true):

Fixture	Chunks	Graph defs	Reconstructed key matched
shapes.rs (Rust)	4	4	4
app.py	5	4	4
greeter.ts	4	3	3
Widget.java	1	5	1
api.ts (arrow fn, interface, decorator)	5	7	4
Svc.java (generics, ctor, enum, record)	2	4	2

Two honest observations:

• Where both passes found the same named symbol, the arithmetic held. I found no off-by-one drift
  in any fixture. Reconstruction is not broken in the way I expected.
• It fails on membership and on missing names. arrowFn is chunked anonymously so its key is
  api.ts#1:function against a node id of api.ts#1:arrowFn; Repo/find/every Java method are
  graphed with no chunk at all. Details in [Ticket]: Chunk and graph symbol sets diverge on every tags language, and nothing tests that they agree #11 and [Ticket]: Java is chunked at class granularity: methods, constructors, interfaces, enums and records produce no chunk #10.

So the reconstruction is mostly right, silently partial, and has no test anywhere holding it that
way. That combination is the argument for making it explicit rather than leaving it inferred.

Proposed change

pub struct Chunk {
    // ...existing fields unchanged...
    /// The graph node this chunk is the body of, when the walk built a graph and this chunk
    /// corresponds to a definition the graph pass also found. `None` for windowed chunks, PDF
    /// text, and any chunk with no corresponding definition node.
    pub node_id: Option<String>,
}

Populated inside walk_checkout, where both passes already see the same parse tree — not derived by
the caller from public fields.

Option rather than a hard guarantee is deliberate, and matches the precision-favouring policy the
README already states for unresolvable calls: leave it None rather than guess. A windowed chunk
has no definition; an ambiguous match should resolve to None, not to a plausible-looking node id. A
chunk with node_id: None is still fully searchable — it just has no structural anchor.

Note this composes with #11 rather than depending on it: fixing the symbol-set drift raises the hit
rate, but the field is worth having either way, because None is then a known answer instead of a
silent miss.

Acceptance

• Chunk::node_id is populated for definition-derived chunks when build_graph is true.
• It is None — never a guess — for windowed chunks, PDF text, and ambiguous matches.
• It is None for every chunk when build_graph is false (no graph, no ids).
• A test asserts, on a tags language (not only Rust), that chunks[i].node_id resolves to an
  actual node in graph.nodes. Rust is the case that already works and would not catch a
  regression here.
• Determinism holds: two walks of the same checkout produce identical node_id assignments.

Out of scope

• Changing how either pass discovers symbols ([Ticket]: Java is chunked at class granularity: methods, constructors, interfaces, enums and records produce no chunk #10, [Ticket]: Chunk and graph symbol sets diverge on every tags language, and nothing tests that they agree #11).
• Any opinion about where a host stores the result. The crate stays a pure extractor; this is one
  extra field on a value it already returns.

Provenance

Found while designing an integration that consumes chunks and graph together and needs to join them —
the join turned out to be the caller's problem, and a partly-silent one.

AI usage: drafted with Claude (Claude Code). The match-rate table is verbatim output from running the
unmodified crate against main @ 293d12f, not an estimate. The "no off-by-one drift" row contradicts
what I predicted before running it and is reported as measured.