charlieviettq/awesome-agent-skill / .claude/skills/algo-nlp-lda
"algo-nlp-lda"
Verified"Implement LDA topic modeling to discover latent topics in document collections. Use this skill when the user needs to extract topics from a text corpus, categorize documents by theme, or explore thematic structure — even if they say 'what are the main topics', 'topic extraction', or 'document clustering by theme'.".
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 4.1 KB |
| examples/sample_scenario.md | markdown | 6.7 KB |
| references/advanced-lda.md | markdown | 12.2 KB |
| references/topic-evaluation.md | markdown | 9.1 KB |
examples/sample_scenario.md
View on GitHub我直接用 Write tool 建立檔案(這是本地工作目錄的操作,不需要 QMD):
The file content to write to `algo-nlp-lda/examples/sample_scenario.md`:
```markdown
# Example: 電商平台用戶評論主題發現
## Scenario
**公司:** TrendMart(台灣中型跨境電商,月訂單 20 萬筆)
**日期:** 2026-03-15
**提問者:** 產品分析師 Lisa
TrendMart 在過去 18 個月累積了 **42,000 筆繁體中文用戶評論**,橫跨美妝、3C、居家、服飾四大品類。客服主管抱怨「每週要手動讀 500 筆差評,根本讀不完」。Lisa 想自動發現評論裡重複出現的主題,讓客服能按主題優先處理,而不是逐筆閱讀。
Lisa 的原始問題:「我有一堆評論,可以幫我抓出主要的抱怨主題嗎?」
---
## Analysis
### Phase 1: Input Validation(前處理)
**原始語料統計:**
- 文件數:42,000 筆評論
- 平均字數:87 字(最短 5 字,最長 632 字)
- 日期範圍:2024-09-01 ~ 2026-03-01
**清洗步驟:**
1. 移除短評(< 15 字)→ 剩 38,200 筆(短文 LDA 可靠性差)
2. 繁體中文斷詞:使用 jieba + 自定義詞典(加入「退款」「包裝破損」「色差」等電商詞)
3. 停用詞移除:「的」「了」「是」「很」「真的」「感覺」等 280 個詞
4. 濾除低頻詞(出現 < 5 篇)與高頻詞(出現 > 40% 文件)
**最終 DTM:**
- 有效文件:38,200 筆
- 詞彙量:11,340 個 term
- ✅ Gate passed:DTM 乾淨,詞彙量在合理範圍(1K-50K)
---
### Phase 2: Core Algorithm(LDA 執行)
**K 值選擇:**
- 啟發公式:√(38200/2) ≈ 138 → 過多,電商評論主題有限
- 實際掃描範圍:K = 5, 8, 10, 12, 15, 20
- Coherence (C_v) 結果:
| K | C_v 平均 | 備註 |
|----|---------|------|
| 5 | 0.41 | 主題過於粗糙,美妝+服飾混在一起 |
| 8 | 0.51 | 明顯提升 |
| **10** | **0.58** | **最佳拐點** |
| 12 | 0.56 | 微降,出現重複主題 |
| 15 | 0.49 | 主題開始碎裂 |
**決定 K = 10**(coherence 最高,且 10 個主題對客服團隊認知負擔合理)
**超參數設定:**
- α = 50/10 = 5.0(允許每篇評論混合多主題)
- β = 0.01
- Gibbs sampling:1,500 iterations,burn-in 500
**多次執行穩定性檢查:**
以 3 個不同 random seed 跑 3 次,top-5 詞彙重疊率 > 85% → 主題穩定。
---
### Phase 3: Verification(主題詮釋)
**10 個主題的 top words 與人工標籤:**
| Topic ID | Top 10 詞彙 | 人工標籤 | Coherence |
|----------|------------|---------|-----------|
| 0 | 物流、配送、快遞、到貨、延遲、包裹、運送、時效、寄送、收到 | 物流與配送 | 0.68 |
| 1 | 色差、顏色、照片、實物、差異、圖片、描述、誤差、拍照、偏色 | 商品色差/描述不符 | 0.62 |
| 2 | 品質、做工、材質、耐用、質感、縫線、料子、用料、手感、剪裁 | 產品品質 | 0.61 |
| 3 | 退款、退貨、客服、處理、申請、回覆、態度、溝通、售後、解決 | 售後服務/退換貨 | 0.59 |
| 4 | 包裝、外盒、紙箱、破損、壓扁、保護、收到、外觀、凹陷、氣泡 | 包裝品質 | 0.57 |
| 5 | 尺寸、尺碼、版型、偏小、偏大、合身、對照、建議、加大、試穿 | 尺寸/版型 | 0.55 |
| 6 | 香味、氣味、成分、過敏、刺激、保濕、效果、膚質、敏感、功效 | 美妝/保養效果 | 0.60 |
| 7 | 螢幕、電池、充電、功能、操作、系統、連線、藍牙、設定、故障 | 3C 產品功能 | 0.63 |
| 8 | 價格、便宜、划算、值得、CP值、比較、促銷、優惠、折扣、性價比 | 價格感知 | 0.53 |
| 9 | 味道、食材、新鮮、口感、過期、包裝、食品、效期、保存、添加 | 食品/保存期限 | 0.56 |
**Coherence 平均:0.574** → 可接受(行業經驗值 0.45-0.65 為合格)
✅ Gate passed:10 個主題均可人工詮釋,無「垃圾主題」(top words 全為無意義詞彙)
---
### Phase 4: 文件分配
每篇評論取 dominant topic(比例最高的主題)作為主分類:
| 主題 | 文件數 | 占比 |
|------|--------|------|
| 物流與配送 | 7,840 | 20.5% |
| 商品色差/描述不符 | 5,920 | 15.5% |
| 產品品質 | 5,350 | 14.0% |
| 售後服務/退換貨 | 4,580 | 12.0% |
| 包裝品質 | 3,820 | 10.0% |
| 尺寸/版型 | 3,440 | 9.0% |
| 美妝/保養效果 | 2,670 | 7.0% |
| 3C 產品功能 | 2,290 | 6.0% |
| 價格感知 | 1,910 | 5.0% |
| 食品/保存期限 | 380 | 1.0% |
---
## Result
```json
{
"topics": [
{"id": 0, "label": "物流與配送", "top_words": ["物流","配送","快遞","到貨","延遲","包裹","運送","時效","寄送","收到"], "coherence": 0.68},
{"id": 1, "label": "商品色差/描述不符", "top_words": ["色差","顏色","照片","實物","差異","圖片","描述","誤差","拍照","偏色"], "coherence": 0.62},
{"id": 2, "label": "產品品質", "top_words": ["品質","做工","材質","耐用","質感","縫線","料子","用料","手感","剪裁"], "coherence": 0.61},
{"id": 3, "label": "售後服務/退換貨", "top_words": ["退款","退貨","客服","處理","申請","回覆","態度","溝通","售後","解決"], "coherence": 0.59},
{"id": 4, "label": "包裝品質", "top_words": ["包裝","外盒","紙箱","破損","壓扁","保護","收到","外觀","凹陷","氣泡"], "coherence": 0.57}
],
"doc_topics": [
{"doc_id": "r_00142", "dominant_topic": 0, "topic_distribution": [0.71, 0.08, 0.05, 0.09, 0.07]},
{"doc_id": "r_00891", "dominant_topic": 1, "topic_distribution": [0.04, 0.65, 0.12, 0.10, 0.09]},
{"doc_id": "r_02033", "dominant_topic": 3, "topic_distribution": [0.06, 0.07, 0.11, 0.68, 0.08]}
],
"metadata": {
"K": 10,
"coherence_avg": 0.574,
"documents": 38200,
"vocabulary": 11340,
"iterations": 1500,
"random_seeds_tested": 3,
"stability": "high (>85% top-5 overlap)"
}
}
```
**給 Lisa 的行動建議:**
1. **優先處理「物流與配送」(20.5%,7,840 筆)**:佔比最高,且多為時效抱怨,可直接與物流商 SLA 對焦。
2. **「商品色差/描述不符」(15.5%)** 是可系統性解決的問題:要求賣家提供色彩校正圖,可預期降低此類投訴。
3. **注意:LDA 主題標籤是分析師根據 top words 人工命名的**,不是演算法自動產生。若後續有新人接手,需重新確認標籤定義。
4. **季節性追蹤**:建議每季重跑一次 LDA,因促銷季(雙11、春節)會產生臨時性主題(如「贈品」「限量」),靜態 LDA 無法捕捉動態主題。
```
以上是完整的 `sample_scenario.md` 內容。請確認後我可以直接寫入 `algo-nlp-lda/examples/sample_scenario.md`。
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Unverified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |