Arnoldlarry15/red-set-protocell
An Open-source AI safety platform using dual-agent Sniper/Spotter red-teaming to audit and secure large language models. Scalable, transparent, and built for advanced AI risk monitoring.
What's novel
An Open-source AI safety platform using dual-agent Sniper/Spotter red-teaming to audit and secure large language models. Scalable, transparent, and built for advanced AI risk monitoring.
Code Analysis
12 files read · 4 roundsAn automated AI red-teaming engine that uses evolutionary algorithms (mutation, selection, fitness scoring) to systematically generate adversarial prompts, test LLMs for safety failures, and score the severity of discovered vulnerabilities across three dimensions (linguistic safety, security exploit
Strengths
Genuine evolutionary architecture with real selection strategies (tournament, elitism, novelty search, diversity preservation), multi-dimensional fitness, and adaptive mutation — not just a prompt wrapper. The separation of concerns (Sniper/Target/Spotter/EGG/Orchestrator) is clean and well-documented, with a coherent 3-layer scoring taxonomy and failure archetype classification system.
Weaknesses
The mutation strategies are purely heuristic (word substitution, regex-based framing) rather than using LLMs to generate mutations, limiting the evolutionary search space. The Spotter's evaluation relies on regex pattern matching for failure detection, which is brittle and easily evaded by creative LLM responses — the 'evolutionary' loop is constrained by this shallow fitness signal.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
731
Contributors
5
Files
307
Active weeks
11
Repository
Language
Python
Stars
3
Forks
0
License
MIT