maestrosalah-dev/relational-time-engine
Relational Time Engine (RTE): runtime density regulation for compute-efficient transformer inference. Demonstrates up to 75% layer reduction with improved latency and throughput.
What's novel
Relational Time Engine (RTE): runtime density regulation for compute-efficient transformer inference. Demonstrates up to 75% layer reduction with improved latency and throughput.
Code Analysis
19 files read · 4 roundsA collection of monkey-patched early-exit wrappers for DistilBERT and GPT-2 that stop layer execution when inter-layer hidden-state drift falls below a threshold, alongside a disconnected rule-based discrimination engine that is never integrated with the model wrappers.
Strengths
The DistilBERT and GPT-2 gate wrappers are functional and demonstrate a working early-exit mechanism on real HuggingFace models with measurable latency improvements. The benchmark scripts are reasonably thorough with multiple runs, warmup, and CSV output.
Weaknesses
The core 'RTE engine' (rte/core.py) with its adaptive threshold, sigmoid gating, and windowed statistics is completely disconnected from the model wrappers — they are two separate systems that never interact. The project lacks real tests (test files are empty), has no KV cache support for autoregressive generation, and the main README benchmark actually uses a static early-exit toy transformer rather than the dynamic RTE mechanism.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
71
Contributors
1
Files
61
Active weeks
2
Repository
Language
Python
Stars
1
Forks
0
License
Apache-2.0