samsaeed22/kevlar-benchmark
🔒 Automate detection and exploitation of Agent-Specific Injection vulnerabilities using the OWASP Top 10 framework for AI agent security testing.
What's novel
🔒 Automate detection and exploitation of Agent-Specific Injection vulnerabilities using the OWASP Top 10 framework for AI agent security testing.
Code Analysis
24 files read · 5 roundsA Python library of hardcoded single-turn prompt-injection test scripts with regex-based output detection, organized around the OWASP Agentic Security Incidents taxonomy, that tests against mock/simulated AI agents designed to be vulnerable by construction.
Strengths
The attack/detector/orchestrator pattern per ASI category provides a clean conceptual framework, and the seccomp-based RCE sandbox in C is the only technically substantive component.
Weaknesses
Multiple files have syntax errors and missing imports that prevent execution, 6 of 10 ASI categories are empty stubs, all 'agents' are hardcoded to be vulnerable, and detection is limited to literal string matching against attacker.com.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
21
Contributors
2
Files
85
Active weeks
5
Repository
Language
Python
Stars
1
Forks
0
License
MIT