Description
Users often ask vague or conversational questions that don't translate well directly to a vector search. The retrieved context quality suffers as a result.
Implementation
- Implement a Query Reformulation step in
bots/: before hitting the vector database, pass the user's query through a fast, cheap LLM to generate optimized search terms or expand the query.
- Implement a Re-ranking step: retrieve the top 20 chunks, then use a Cross-Encoder model (like
ms-marco-MiniLM-L-6-v2) to re-score and select the top 5 most relevant chunks to feed to the final generation LLM.
- This will significantly improve answer accuracy for complex documents.
Level: Advanced
Affected Files: bots/retriever.py, backend/llm_utils.py
Description
Users often ask vague or conversational questions that don't translate well directly to a vector search. The retrieved context quality suffers as a result.
Implementation
bots/: before hitting the vector database, pass the user's query through a fast, cheap LLM to generate optimized search terms or expand the query.ms-marco-MiniLM-L-6-v2) to re-score and select the top 5 most relevant chunks to feed to the final generation LLM.Level: Advanced
Affected Files:
bots/retriever.py,backend/llm_utils.py