A production-oriented Retrieval Augmented Generation (RAG) system designed to help employees query internal knowledge bases using natural language. The solution runs locally with Ollama, requires no API keys, and is built to be accurate, grounded, and practical for enterprise use.
Enterprise Knowledge Assistant
Built by: Vidhi Waghela
Enterprises often store critical knowledge across HR policies, technical guides, compliance documents, customer FAQs, and product documentation. Searching these files manually is slow and inefficient. This project addresses that problem by building an AI-powered assistant that can:
- ingest internal documents,
- extract and index semantic knowledge,
- retrieve the most relevant context,
- generate grounded answers,
- and cite the source documents used for each response.
The system is designed as a practical RAG application with strong engineering discipline: local-first deployment, retrieval quality control, prompt grounding, source attribution, and a simple user interface.
Employees regularly ask questions such as:
- “What is the leave policy?”
- “How many sick leaves are allowed?”
- “What is the refund procedure?”
- “What does the SLA guarantee?”
Manually searching through large internal repositories is time-consuming and error-prone. The Enterprise Knowledge Assistant provides a faster and more reliable method for retrieving information from organization documents.
This solution delivers the core capabilities required for the assignment:
- Document ingestion and preprocessing
- Knowledge extraction and indexing
- Semantic retrieval
- Context-aware answer generation
- Source citation and traceability
- User interaction through a web app, API, and CLI
┌──────────────────────────────────────────────────────────────────────┐
│ ENTERPRISE KNOWLEDGE ASSISTANT │
├──────────────────────────────────────────────────────────────────────┤
│ │
│ ┌───────────────┐ ┌───────────────┐ ┌──────────────────────┐ │
│ │ Streamlit UI │ │ FastAPI API │ │ CLI │ │
│ │ Chat / Upload│ │ /ask /ingest │ │ ingest / ask / eval │ │
│ └──────┬────────┘ └──────┬────────┘ └──────────┬───────────┘ │
│ └──────────────────┴─────────────────────────────┘ │
│ │ │
│ ┌──────────▼──────────┐ │
│ │ RAG Pipeline │ │
│ │ retrieve + generate │ │
│ └──────────┬──────────┘ │
│ │ │
│ ┌───────────────────────┼───────────────────────┐ │
│ │ │ │ │
│ ┌─────▼──────┐ ┌──────▼────────┐ ┌──────▼────────┐ │
│ │ Hybrid Search│ │ FAISS Store │ │ Ollama LLM │ │
│ │ Semantic + BM25│ │ Vector Index │ │ llama3.2:3b │ │
│ └─────────────┘ └────────────────┘ └──────────────┘ │
│ │
│ ┌────────────────────────────────────────────────────────────────┐ │
│ │ Document Ingestion Pipeline: Files → Extract → Chunk → Embed │ │
│ │ Supported formats: PDF, TXT, MD, DOCX, CSV │ │
│ └────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────┘
User Question
│
▼
Query embedding via Ollama
│
▼
Semantic retrieval from FAISS index
│
▼
BM25 keyword scoring on retrieved candidates
│
▼
RRF fusion to combine semantic and keyword signals
│
▼
Context assembly with source metadata
│
▼
Grounded answer generation with local LLM
│
▼
Answer + citations + confidence score
Document files
│
▼
Text extraction
│
▼
Chunking
│
▼
Embedding generation
│
▼
FAISS indexing + metadata persistence
| Component | Technology | Why it was chosen |
|---|---|---|
| LLM | Ollama + llama3.2:3b | Fully local, privacy-preserving, fast enough for enterprise demos |
| Embeddings | Ollama + nomic-embed-text | Retrieval-optimized embedding model that runs locally |
| Vector Store | FAISS | Lightweight, dependency-light, efficient exact similarity search |
| API Layer | FastAPI | High-performance async API with automatic documentation |
| Web UI | Streamlit | Fast to build, intuitive for demos and interactive testing |
| CLI | Python | Simple command-line workflow for ingest, ask, eval, and stats |
| Document Parsing | pdfplumber / python-docx / text readers | Supports common enterprise document formats |
The system is designed to run 100% locally using Ollama. This is critical for enterprise environments where internal policy and compliance documents cannot be sent to third-party APIs.
The model is instructed to answer only from the provided retrieved context. This significantly reduces hallucinations and makes the answers more trustworthy.
The system combines:
- semantic search using embeddings,
- BM25 keyword scoring,
- and Reciprocal Rank Fusion (RRF)
This improves recall for both conceptual and exact-match questions.
FAISS was chosen because it avoids the sqlite dependency issue on macOS and offers efficient, exact nearest-neighbor retrieval for this use case.
Documents are split into smaller chunks with overlap to preserve context across boundaries while keeping retrieval precise.
Each response includes a confidence score and source references so users can verify the answer and understand where it came from.
- Local document ingestion from multiple file types
- Semantic search over embedding vectors
- Hybrid retrieval using semantic + keyword signals
- Source-grounded answer generation
- Confidence scoring for retrieval quality
- Web UI through Streamlit
- REST API for integration
- CLI for batch workflows and evaluation
- Optional query rewriting for improved retrieval
- Python 3.9+
- Ollama installed and running
ollama pull llama3.2:3b
ollama pull nomic-embed-textpython3 -m venv venv
source venv/bin/activatepip install -r requirements.txtollama servestreamlit run streamlit_app.pyThen open the local browser URL shown by Streamlit.
uvicorn app.api:app --reload --host 0.0.0.0 --port 8000API docs will be available at:
http://localhost:8000/docs
python cli.py ingest ./data/sample_docs
python cli.py ask "What is the leave policy?"
python cli.py interactive
python cli.py stats
python cli.py evalcurl -X POST http://localhost:8000/ask \
-H "Content-Type: application/json" \
-d '{"question": "What is the refund policy?"}'Example response:
{
"answer": "Customers are eligible for a full refund within 30 days of purchase. (Source 1)",
"sources": [
{
"document": "Customer_Policy.md",
"page": 1,
"relevance_score": 0.847
}
],
"confidence": 0.847,
"confidence_label": "high",
"chunks_used": 3,
"question": "What is the refund policy?"
}curl -X POST http://localhost:8000/ingest -F "files=@HR_Policy.pdf"curl http://localhost:8000/statscurl http://localhost:8000/healthThe project includes an evaluation workflow for testing answer quality and retrieval behavior.
- accuracy of retrieved information,
- relevance of generated answers,
- citation quality,
- and hallucination prevention.
- smaller retrieval context to reduce noise,
- lower temperature for more deterministic answers,
- local embedding cache for repeated queries,
- and strict grounding instructions for safer enterprise use.
- Sequential embedding can be slower for very large document sets.
- FAISS does not provide native deletion semantics like a full vector database.
- OCR for scanned PDFs is not included.
- Table-heavy PDFs may lose structural fidelity during extraction.
- The system is optimized for English-language documents.
- add re-ranking for higher retrieval precision,
- support Docker deployment,
- introduce authentication and RBAC,
- improve multi-document reasoning,
- add user feedback collection,
- and integrate a more advanced evaluation framework such as RAGAS.
enterprise-knowledge-assistant/
├── app/
│ ├── api.py
│ ├── ingestion.py
│ ├── rag.py
│ └── vector_store.py
├── data/
│ ├── sample_docs/
│ └── chroma_db/
├── tests/
├── cli.py
├── config.py
├── streamlit_app.py
├── requirements.txt
└── README.md
This project demonstrates a strong end-to-end RAG implementation for an enterprise knowledge assistant. It combines modern AI retrieval techniques with practical software engineering principles to create a system that is local, grounded, explainable, and suitable for real-world internal knowledge search scenarios.