Update Vector Graph RAG with two-stage Jev evaluation

[zc277584121]

Vector Graph RAG now uses two-stage Jev relation selection and full-passage reranking. Its evaluation covers 1,000 MuSiQue and 1,000 2Wiki queries, reaching 76.78% and 95.35% Recall@5 respectively (86.07% average).

The project reports its strongest retrieval results among the compared baselines. The linked report includes saved results, reproduction scripts, and the quality–latency comparison.

Evaluation: https://github.com/zilliztech/vector-graph-rag/blob/main/evaluation/jev/two-stage/README.md

I am a contributor to Vector Graph RAG and work on search at Zilliz.

Updates the existing entry JSON and its implementation/evaluation evidence paths. README is left to the repository's generation workflow.

Validation: npm run check (180 entries), npm test (4 tests), and git diff --check passed.

[github-actions]

Jev review

Needs maintainer review

GitHub listing (not a review criterion): 255 stars · organization zilliztech · 1056 followers

Suggested category: search · category confidence: 0.89

Criterion	Probability of yes	Result
Concrete Jev integration	0.93	pass
Description supported by evidence	0.90	pass
Usable setup documentation	0.94	pass

Follow-up:

• Full file omitted because it exceeds the evidence budget: src/vector_graph_rag/rag.py; maintainer review required
• Full file omitted because it exceeds the evidence budget: src/vector_graph_rag/graph/retriever.py; maintainer review required
• Full file omitted because it exceeds the evidence budget: evaluation/jev/plot_retrieval.py; maintainer review required

Evidence inspected:

• README.md
• src/vector_graph_rag/llm/jev.py
• evaluation/jev/two-stage/README.md
• pyproject.toml
• frontend/src/api/queries.ts
• evaluation/jev/two-stage/summarize.py
• evaluation/jev/two-stage/plot.py

PR commit: d7814296cef68ed5aa8ed37a6982cf66c26f95e8
Model: typesafe-ai/jev via vercel · policy: 9675860e2c86 · input tokens: 16034
Review run and JSON report

Model judgments use Jev only; this comment is generated from a template. Probabilities are model judgments, not verified accuracy. A maintainer decides whether to merge.

[zc277584121]

Hi @fatwang2, the automated review passed the three criteria but requested maintainer review because some source files exceeded its evidence budget. This updates the existing Vector Graph RAG entry for two-stage Jev relation selection and passage reranking, with 86.07% average Recall@5 across 2,000 queries. The linked evaluation and Jev implementation provide the supporting details. Could you take a look when convenient? Thanks!

[fatwang2]

Maintainer review of d781429: approved.

At zilliztech/vector-graph-rag@e32c077 I inspected both the Jev client and the full rag.py omitted by automated discovery. Relations are scored with Noul questions and stably selected; graph and dense candidates are merged, then full passages receive a second Noul scoring pass in the actual retrieval flow. The implementation, MIT license, setup and search category support this update.

I independently recomputed Recall@5/10 from all 10,000 saved method/query records, checked all ten groups against the manifest's 1,000 query IDs per dataset, and matched summary.json. Two-stage Jev has 76.783333% MuSiQue and 95.35% 2Wiki Recall@5; the equal-weight average is 86.066667%, rounding to the entry's 86.07%. The entry says “reporting” and is scoped to those datasets. This verifies the saved-result arithmetic, not a fresh model run or complete historical candidate/model-call replay; the report documents that limitation and prior exploration.

Catalog CI and the Jev workflow passed. Local catalog validation/build and all four combined catalog tests passed.