Skip to content

feat(index): resolve cross-file references via a project-wide symbol table - #56

Merged
r0h1tb merged 2 commits into
mainfrom
feat/global-symbol-resolution
Aug 2, 2026
Merged

feat(index): resolve cross-file references via a project-wide symbol table#56
r0h1tb merged 2 commits into
mainfrom
feat/global-symbol-resolution

Conversation

@r0h1tb

@r0h1tb r0h1tb commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Refs #35

This is the step-4 work I proposed in my analysis on #35 — the piece that turns the call graph from "same file only" into something usable.

Stacked on #50. That PR fixes for n in [], without which no CALLS edge is emitted at all and none of this is observable. Review #50 first; this branch contains it.

Problem

EdgeExtractor.extract_edges resolved references against:

name_to_id = {n.name: n.id for n in nodes}

nodes is the node list for the file currently being parsed, so a callee defined in another file was never in the map and the edge was dropped.

I sized the gap using ground truth from parsers with no shared code with tree-sitter — Python's stdlib ast here, and javalang on a Spring Boot service — so the system isn't graded against its own output:

codebase intra-file calls cross-file unreachable
this repo 507 2,521 83%
Spring Boot service 116 2,063 95%

In layered Java (controller → service → repository) essentially every interesting call crosses a file, which is why the Java number is so stark.

Approach

Indexing becomes two-phase:

  1. Collect — parse every file, extract nodes, build a project-wide name -> id map
  2. Resolve — walk the parsed files again, extracting edges against that map

extract_edges takes an optional global_symbols, and the local map is applied last:

name_to_id = dict(global_symbols or {})
name_to_id.update({n.name: n.id for n in nodes})

so a file-local symbol always shadows a same-named symbol from elsewhere. Omitting global_symbols preserves the old behaviour exactly, so index-folder and any other caller are unaffected.

First definition of a name wins in the global map, and files are walked in a stable order, so the choice is deterministic across runs.

Results on this repo

before after
CALLS edges 498 1,412
of which cross-file 0 914
total edges 2,387 3,782

Retrieval quality over the 15 most-called symbols, scored against the stdlib ast oracle across all call relationships rather than just same-file ones:

symbol                      expected   raged      P      R     F1
run                               93      93   1.00   1.00   1.00
session                           82      82   1.00   1.00   1.00
_make_sentinel_tree               37      34   0.97   0.89   0.93
ParserManager                     34      34   1.00   1.00   1.00
ASTEdge                           25      15   1.00   0.60   0.75
...
MEAN                                           0.99   0.95   0.97

Precision holding at 0.99 is the load-bearing result. Name-based global resolution could plausibly have produced false positives across same-named symbols in different files; on this codebase it does not. run and session are heavily cross-file and both score 1.00.

Recall isn't 1.00 — ASTEdge at 0.60 is the weakest. Those are attribute-style call sites the calls query doesn't capture, which is a separate gap from resolution and I left it alone.

Known limitation

Resolution is by bare name, not qualified name. Two distinct Config.load() in different modules will collapse to whichever was indexed first. That didn't bite on this repo (precision 0.99), but it will on a codebase with heavy name reuse.

The principled fix is qualified-name resolution with a bare-name fallback, plus a lower confidence on fallback matches — the edge model already carries confidence. I've deliberately not done that here so the change stays reviewable and the measurement stays interpretable. Happy to follow up if you want it before this lands.

Tests

4 in tests/test_global_symbol_resolution.py: the unresolved baseline, cross-file resolution, local-shadows-global, and that an unrelated global table leaves same-file behaviour byte-identical.

Verification

pytest tests/          3 failed, 188 passed, 1 xfailed
ruff check             All checks passed!

Baseline on main is 3 failed, 174 passed — same three pre-existing failures (fixed in #51), plus the new tests.

CI will be red at Lint with ruff until #51 lands; that's the pre-existing breakage, not this change.

r0h1tb added 2 commits August 2, 2026 17:26
…table

Edge extraction resolved references against name_to_id built from a single
file's nodes, so any reference to a symbol defined elsewhere was dropped. Only
same-file calls ever linked.

Measured against ground truth from parsers independent of tree-sitter (Python's
stdlib ast, and javalang for a Java project), the unreachable share was 83% on
this repo and 95% on a Spring Boot service -- in layered code essentially every
interesting call crosses a file.

Indexing is now two-phase: collect nodes from every file and build a project
wide name -> id map, then resolve edges against it. extract_edges takes an
optional global_symbols map; local definitions are applied last so a file-local
symbol always shadows a same-named symbol from another file.

Measured on this repo, before -> after:

  CALLS edges           498 -> 1412
  of which cross-file     0 ->  914
  total edges          2387 -> 3782

Retrieval quality over the 15 most-called symbols, scored against the stdlib
ast oracle across all call relationships (not just same-file):

  precision 0.99, recall 0.95, F1 0.97

Precision holding at 0.99 is the load-bearing result: name-based global
resolution could have produced false positives across same-named symbols, and
on this codebase it does not.

Suite: 3 failed, 188 passed (baseline: 3 failed, 174 passed) -- same three
pre-existing failures, fixed separately in #51.

Refs #35
UAT against a live index found 'ast-rag refs' and 'ast-rag symbol-impact'
failing with:

  Neo.ClientError.Statement.ParameterMissing
  Expected parameter(s): call_kinds

Regression from the call-traversal rewrite: two queries were changed to filter
on $call_kinds, but their session.run() calls were never given the parameter.
Neo4j only reports this at execution time, so nothing caught it -- the unit
tests never reach these branches without a populated graph.

Binds the parameter at both sites. After the fix, 'refs ParserManager' returns
its references and 'symbol-impact ParserManager' reports 34 references and
42 callers.

Adds a static checker over the API and repository layers: for every
session.run(<var>, **kwargs) it resolves <var> back to its query text and
asserts each $parameter is bound. Reverting the fix makes it fail, which is
the property the first version of this test lacked -- a runtime test could not
reach the broken branch and passed either way.
@r0h1tb
r0h1tb force-pushed the feat/global-symbol-resolution branch from 305614e to e3557d5 Compare August 2, 2026 12:03
@r0h1tb

r0h1tb commented Aug 2, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto current main. Two things worth flagging, since the rebase wasn't purely mechanical:

1. I resolved a real conflict in ast_rag/cli.py, and it changed the code. This branch still drove the index loop with console.status(...), but #54 landed _index_progress() progress bars on main in the meantime. I kept main's progress bars and put the two-phase index inside them, rather than reverting #54's UI. I also converted this PR's second phase (reference resolution) from console.status to the same progress bar — resolution is exactly the kind of multi-minute phase #54's docstring says should report count/elapsed/ETA, and having phase 1 be a bar and phase 2 a spinner would have looked broken. Shout if you'd rather I left phase 2 as a spinner.

2. ruff format wanted one change in ast_rag_api.py (argument wrapping in the call_kinds commit). Amended in, so --check is clean.

Verified locally against main (6f71a9b):

main this branch
ruff check / ruff format --check pass pass
pytest tests/ 209 passed, 1 skipped, 1 xfailed 216 passed, 1 skipped, 1 xfailed

The +7 are this PR's tests/test_global_symbol_resolution.py. Nothing pre-existing changed state.

Note on ordering: #60 is stacked on this branch, so this one should go in first.

@r0h1tb
r0h1tb merged commit ef8e7f1 into main Aug 2, 2026
1 check passed
@r0h1tb
r0h1tb deleted the feat/global-symbol-resolution branch August 2, 2026 12:09
@github-project-automation github-project-automation Bot moved this from Backlog to Done in raged kanban Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant