feat: add FP16 auto-precision and batching parameters to Reranker #813

[suhaniiz]

📋 PR Checklist

Thank you for contributing to PDF-Assistant-RAG! 🎉
Please fill out this template before submitting. PRs without it filled in will be closed.

---

🔗 Related Issue

Closes #813

---

📝 What does this PR do?

This PR introduces performance and memory optimizations to the Reranker class (app/reranker.py):

1. FP16 Auto-Precision: Automatically detects if a CUDA-enabled device is available and configures the CrossEncoder model initializer with torch_dtype=torch.float16. This reduces GPU memory overhead and accelerates inference.
2. Batching Control: Adds a batch_size parameter (defaulting to 32) to the rerank() signature and forwards it directly to the underlying model.predict() function. This prevents CUDA Out-Of-Memory (OOM) exceptions when ranking a massive volume of document chunks simultaneously.

---

🗂️ Type of Change

• 🐛 Bug fix
• ✨ New feature
• 🔧 Refactor / code cleanup
• 📝 Documentation update
• 🎨 UI / styling change
• ⚙️ CI / tooling / config change
• 🧪 Tests

---

🧪 How was this tested?

• Tested the affected API endpoints manually
• Ran the backend locally (uvicorn app.main:app --reload)
• Ran the frontend locally (npm run dev inside frontend/)
• Added / updated tests

---

📸 Screenshots (if UI change)

---

⚠️ Anything to flag for reviewers?

The torch dependency was safely imported dynamically inside _load_model() to minimize runtime impact if the file is imported elsewhere in environments missing PyTorch. No other core systems are impacted by these optimizations.

---

✅ Self-Review Checklist

• My branch is based on dev, not main
• I have not added any secrets / API keys
• I have not modified main branch or any HuggingFace deployment config
• My code follows the existing style (no unnecessary formatting changes)
• I have updated relevant docs / comments if needed

[vercel]

@suhaniiz is attempting to deploy a commit to the param20h's projects Team on Vercel.

A member of the Team first needs to authorize it.

[suhaniiz]

@param20h , kindly review this PR which is under GSSoC 2026

[suhaniiz]

@param20h