d-oit/web-doc-resolver
Resolve a query or URL into compact, LLM-ready markdown using a low-cost cascade: llms.txt first, Exa highlights, Tavily fallback, Firecrawl last.
What's novel
Resolve a query or URL into compact, LLM-ready markdown using a low-cost cascade: llms.txt first, Exa highlights, Tavily fallback, Firecrawl last.
Code Analysis
17 files read · 5 roundsResolves web URLs and search queries into compact LLM-ready Markdown by cascading through tiered providers (free to paid) with semantic caching, quality scoring, circuit breakers, and budget-aware routing.
Strengths
Genuinely well-architected multi-provider cascade with real depth: semantic vector cache via sqlite-vec, budget-aware routing with platform detection, circuit breakers, negative caching, and quality scoring with bot-challenge detection. The async race pattern with per-provider latency thresholds from learned routing memory is a thoughtful design.
Weaknesses
The cascade engine has a dead code path (found_final is never set to True), the Mistral browser provider creates and destroys an agent per call which is wasteful, and the provider implementations duplicate sync/async logic extensively rather than having one canonical implementation. The project is also sprawling across Python, Rust, and Next.js with significant overlap.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
82
Contributors
3
Files
71
Active weeks
2
Repository
Language
Python
Stars
1
Forks
0
License
MIT