Ashik245-commits/LLM-Filter-Probe
🕵️♂️ Analyze and reverse engineer keyword filtering in large language models to enhance compliance and operational insights.
What's novel
🕵️♂️ Analyze and reverse engineer keyword filtering in large language models to enhance compliance and operational insights.
Code Analysis
10 files read · 3 roundsA tool that reverse-engineers LLM API gateway content filters by using binary search and bidirectional squeeze algorithms to precisely identify which keywords/phrases trigger blocking, with dynamic masking to minimize API calls.
Strengths
The core algorithm is genuinely well-designed: the hybrid binary-search-to-precision-squeeze pipeline, iso-length lazy masking to maintain coordinate consistency, and the 'find once, mask globally' optimization show deep thought about the problem. Error handling is thorough with explicit distinction between network errors and safe responses, retry logic via tenacity, and multiple fallback strategies in the precision scanner.
Weaknesses
No test files exist in the project despite the complexity of the algorithms, which is a significant gap. The README is misleading — it describes a simple 'enter text, click analyze' desktop app, while the actual code is a sophisticated async web application with a Vue frontend, WebSocket real-time updates, and complex algorithmic scanning. There's also code duplication between scan_strategy.py and binary_searcher.py implementing similar logic.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
24
Contributors
3
Files
115
Active weeks
4
Repository
Language
Python
Stars
1
Forks
0
License
GPL-3.0