mjglatzmaier/mlpipe
A lightweight, DAG-based pipeline framework for ML workflows with YAML configuration and parallel batch execution.
What's novel
A lightweight, DAG-based pipeline framework for ML workflows with YAML configuration and parallel batch execution.
Code Analysis
21 files read · 4 roundsA lightweight DAG-based pipeline framework that lets users define ML workflow stages, wire them via Python dicts or YAML, and execute them in topological order with optional joblib-based parallel batch processing.
Strengths
Clean, well-documented core with proper DAG validation (cycle detection, missing dependency checks), good separation of concerns across core/managers/utils, and honest, working examples that match the README exactly. The code is small but complete and coherent for its stated purpose.
Weaknesses
The AutoTuner is a 20-line random sampler with no real optimization logic, and the overall project is a thin orchestration layer over networkx and joblib rather than implementing novel algorithms. The stage_config attribute is set dynamically without being declared in PipelineContext, and there's no retry/failure handling for individual stages.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
13
Contributors
1
Files
33
Active weeks
2
Repository
Language
Python
Stars
2
Forks
0
License
MIT