kxgst228/rag-forge
Modular RAG framework with hybrid retrieval, intelligent chunking, and multi-provider LLM support
What's novel
Modular RAG framework with hybrid retrieval, intelligent chunking, and multi-provider LLM support
Code Analysis
11 files read · 3 roundsSystematically benchmarks RAG pipeline configurations by running all combinations of 4 chunking strategies, 3 embedding models, 3 retrieval methods, and 2 rerankers against a QA dataset, then ranks them by hit rate, MRR, and latency.
Strengths
Excellent separation of concerns with each pipeline stage in its own module, smart caching to avoid redundant computation across the 72-config grid, and remarkably honest documentation that accurately describes limitations (string-containment evaluation, paragraph-based 'semantic' chunking, hardcoded models).
Weaknesses
The evaluation is a crude substring match rather than true semantic relevance, the benchmark loop is sequential with no parallelism, and the 'semantic' chunker is just paragraph merging with no actual semantic understanding despite the name.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
15
Contributors
0
Files
22
Active weeks
9
Repository
Language
Python
Stars
1
Forks
0
License
MIT