chayuto/thai-job-nlp-ner
Efficient Named Entity Recognition (NER) for informal Thai job postings. This project fine-tunes WangchanBERTa and PhayaThaiBERT to extract structured HR data from unstructured social media text without the latency or cost of LLM APIs.
What's novel
Efficient Named Entity Recognition (NER) for informal Thai job postings. This project fine-tunes WangchanBERTa and PhayaThaiBERT to extract structured HR data from unstructured social media text without the latency or cost of LLM APIs.
Code Analysis
12 files read · 3 roundsFine-tunes Thai BERT models (PhayaThaiBERT/WangchanBERTa) for 7-class NER on informal Thai job postings, with a TCC-aware fuzzy alignment pipeline to convert silver-label entity spans into IOB2 training data, served via FastAPI.
Strengths
The TCC-aware fuzzy matching and offset_mapping-based token alignment are genuinely non-trivial domain-specific engineering that solves real problems with Thai text and WangchanBERTa's tokenizer quirks. Clean modular architecture with clear separation between data loading, alignment, training, inference, and evaluation.
Weaknesses
Zero test coverage — no unit tests, no integration tests, no test files at all. The template-based synthetic data generator is crude and the overall approach (fine-tune BERT for NER) is standard, with the novelty concentrated in the alignment layer rather than the model itself.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
29
Contributors
1
Files
69
Active weeks
2
Repository
Language
Python
Stars
1
Forks
0
License
MIT