ashita-ai/arbiter — IdeaCred

Arbiter provides simple APIs, complete observability, and provider-agnostic infrastructure for evaluating LLM outputs. Built on PydanticAI with automatic interaction tracking, multiple evaluators, and extensible architecture.

What's novel

Arbiter provides simple APIs, complete observability, and provider-agnostic infrastructure for evaluating LLM outputs. Built on PydanticAI with automatic interaction tracking, multiple evaluators, and extensible architecture.

Code Analysis

Implements a basic data model for scoring metrics with comparison operators, hashing support, and threshold checking, alongside an LLM interaction wrapper that likely handles API calls to external language models.

Score Breakdown

Innovation

4 (25%)

Craft

79 (35%)

Traction

34 (15%)

Scope

84 (25%)

Signal breakdown

Innovation

Not Fork+1

Code Novelty+1

Concept Novelty+1

Craft

Ci+5

Tests+8

Polish+2

Releases+4

Has License+5

Code Quality+22

Readme Quality+15

Recent Activity+4

Structure Quality+5

Commit Consistency+4

Has Dependency Mgmt+5

Traction

Forks+12

Stars+12

Hn Points+0

Watchers+0

Early Traction+5

Devto Reactions+0

Community Contribs+5

Scope

Commits+8

Languages+8

Subsystems+13

Bloat Penalty+0

Completeness+7

Contributors+8

Authored Files+15

Readme Code Match+3

Architecture Depth+7

Implementation Depth+8

Evidence

Commits

188

Contributors

7

Files

132

Active weeks

8

TestsCI/CDREADMELicenseContributing

Repository

Language

Python

Stars

8

Forks

7

License

MIT