11e3/mer-insight-pipeline
Automated financial prediction tracking & verification using Korean macro data and LLM analysis
What's novel
Automated financial prediction tracking & verification using Korean macro data and LLM analysis
Code Analysis
19 files read · 4 roundsAutomatically extracts verifiable financial predictions from Korean/English blogs, matches them against an 88K+ news headline database using hybrid keyword+vector search, and renders accuracy verdicts via Claude Haiku — with a full eval framework and Streamlit dashboard.
Strengths
Genuinely rigorous engineering: controlled experiments comparing 6 verification approaches, documented data contamination audit with remediation, domain-specific Korean NLP (kiwipiepy POS + financial stopwords), and a two-tier matching system (GIN array overlap → pgvector cosine) that achieves 87% cost reduction over web-search-based verification. The prompt engineering for converting vague blog statements into verifiable yes/no propositions is the real intellectual contribution.
Weaknesses
The verification pipeline is inherently limited by headline availability — predictions about niche assets or long-term forecasts (2026-2028) will remain PENDING indefinitely. The BM25 index uses a naive O(n) argsort instead of numpy, and the hybrid search RRF is applied to insights (not headlines), creating a slight architectural mismatch between the search module and the verification path.
Score Breakdown
Signal breakdown
Innovation
Craft
Traction
Scope
Evidence
Commits
140
Contributors
3
Files
112
Active weeks
1
Repository
Language
Python
Stars
1
Forks
0
License
—