embabel/embabel-agent-experimental
Embabel Agent Experimental Modules (incubated for 2.0)
What's novel
Embabel Agent Experimental Modules (incubated for 2.0)
Code Analysis
28 files read · 7 roundsAn experimental incubation framework for building LLM-powered agents, providing natural-language-to-spec compilation, Docker sandboxed code execution, LLM-as-judge evaluation, and API learning from OpenAPI/GraphQL specs into executable tools.
Strengths
Clean abstraction layers (SandboxedExecutor, NaturalLanguageCompiler, StepSpec) with well-documented interfaces, sealed hierarchies for exhaustive handling, and a coherent multi-module architecture that separates concerns between spec compilation, execution, evaluation, and API integration. The LLM-as-judge eval framework with weighted scoring (timing + subjective + assertions + failure penalty) and the natural-language-to-typed-spec compiler with intermediate type inference are genuinely useful
Weaknesses
The eval runner uses println() for logging instead of the logger, has a hardcoded model (GPT_41_MINI), and the scoring formula's weights (2.0, 4.0, /7.0) are magic numbers without justification. The spec compiler's LLM output parsing is fragile — it relies entirely on the LLM producing valid JSON matching the data class structure, with no schema validation or retry logic beyond a single try/catch. Test coverage appears thin relative to the 40 source files.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
150
Contributors
8
Files
175
Active weeks
33
Repository
Language
Kotlin
Stars
4
Forks
2
License
Apache-2.0