Thisen-Ekanayake/HelaBERT-Analysis
Finetuning and evaluation framework for HelaBERT, enabling Sinhala NLP tasks such as sentiment analysis, news classification, and writing style detection.
What's novel
Finetuning and evaluation framework for HelaBERT, enabling Sinhala NLP tasks such as sentiment analysis, news classification, and writing style detection.
Code Analysis
8 files read · 3 roundsFine-tunes HelaBERT (a Sinhala BERT model) on four text classification tasks (sentiment, news category, news source, writing style) with a novel co-attention classifier head and a context-aware cross-attention architecture for sentiment grounded in article context, evaluated via 5-seed runs.
Strengths
The co-attention mechanism is mathematically sound with proper NaN-safe softmax, mask handling, and a well-reasoned clsconcat variant that strictly generalizes the baseline; the 5-seed sweep with proper train/val/test separation shows methodological rigor and addresses a real cherry-picking concern.
Weaknesses
Extreme code duplication — the same training loop, dataset class, and model head are copy-pasted across 40+ files with only path/hyperparameter changes, making maintenance fragile; there are no unit tests for the core logic (co-attention math, tokenization, dataset construction), and the test/ directory contains legacy code with a different architecture than the finetune/ scripts.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
76
Contributors
1
Files
27
Active weeks
2
Repository
Language
Python
Stars
1
Forks
0
License
—