sgl-project/rbg
A workload for deploying LLM inference services on Kubernetes
What's novel
A workload for deploying LLM inference services on Kubernetes
Code Analysis
8 files read · 3 roundsA Kubernetes operator that orchestrates distributed, multi-role AI inference workloads (e.g., LLM serving with gateway/prefill/decode roles) as a single coordinated unit, providing atomic deployment, coordinated scaling with maxSkew policies, gang scheduling, in-place updates, and topology-aware ser
Strengths
Genuinely novel coordination scaling algorithm that limits role progress divergence via maxSkew with progression gating, plus a well-abstracted gang scheduling layer supporting both scheduler-plugins and Volcano with per-role minimums. The code is exceptionally well-documented with detailed comments explaining design decisions, edge cases, and migration paths.
Weaknesses
The codebase is very large (764 files) with significant complexity in the reconciliation pipeline, making it hard to reason about end-to-end behavior. Some areas like the discovery injector and instance scaling logic were not fully readable due to size, suggesting potential for further decomposition.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
298
Contributors
21
Files
8035
Active weeks
44
Repository
Language
Go
Stars
187
Forks
47
License
Apache-2.0