zero-yx/static_flow / skills/music-ingestion-publisher
music-ingestion-publisher
VerifiedIngest music into StaticFlow Music DB via ncmdump-rs CLI and sf-cli: search/download from Netease Cloud Music or Bilibili, decrypt local NCM files, extract metadata/lyrics/cover art, and publish to lancedb-music.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 4.2 KB |
| flows/bilibili.md | markdown | 3.0 KB |
| flows/common.md | markdown | 3.3 KB |
| flows/local.md | markdown | 896 B |
| flows/netease.md | markdown | 3.4 KB |
flows/common.md
View on GitHub# Common Operations
Shared procedures for all ingestion flows.
## CRITICAL: Never Write Directly to LanceDB
The songs table uses Lance blob v2 encoding for `audio_data`
(`Struct<data: LargeBinary?, uri: Utf8?>`).
Direct writes (Python lancedb, arrow, manual RecordBatch, etc.) **WILL** corrupt
the table — the fragment's `audio_data` column encoding won't match blob v2
struct layout, causing `lance-encoding decoder.rs` errors on read.
**ALWAYS** use `sf-cli write-music` for ingestion.
Binary location:
- Worker payloads: use `sf_cli_path` exactly.
- Manual runs: rebuild `cargo build -p sf-cli --release`, then use `./target/release/sf-cli`.
- Do not replace that with legacy `./bin/sf-cli` or stale `target/*/sf-cli` snapshots for blob v2 tables.
## Verification (Post-Write)
```bash
sf-cli db --db-path /mnt/wsl/data4tb/static-flow-data/lancedb-music \
query-rows songs \
--where "id='<song_id>'" \
--columns id,title,artist,album,album_id,cover_image,format
```
Checklist:
- `title`, `artist`, `album` populated (not "Unknown")
- `album_id` set for Netease tracks
- `cover_image` is a valid `https://` URL (not http, not empty)
- `format` is `mp3` or `flac`
## Cover Image Update
**Recommended**: use `--cover-url` during ingestion (one-step):
```bash
sf-cli write-music \
--db-path /mnt/wsl/data4tb/static-flow-data/lancedb-music \
--file /tmp/music/<file>.mp3 \
--cover-url "https://..." \
... # other flags
```
**Fallback** (post-write update, if cover URL was not available at ingest time):
```bash
sf-cli db --db-path /mnt/wsl/data4tb/static-flow-data/lancedb-music \
update-rows songs \
--where "id='<song_id>'" \
--set "cover_image='<cover_url>'"
```
- Bilibili covers: API returns `http://` — **always convert to `https://`**
(Bilibili CDN supports https; http causes mixed-content blocking).
- Netease covers: already `https://`.
- Accepts full URL or filename (frontend constructs `/api/images/<filename>`).
## Lyrics (Cross-Source)
Bilibili has no lyrics API. For Bilibili-sourced songs, fetch lyrics from
Netease if the original `netease_track_id` is known:
```bash
ncmdump-cli lyric <netease_track_id>
```
Split output at `--- Translation ---` into `.lrc` and `.tlyric.lrc` files.
## Vector Embedding Backfill
`write-music` auto-generates vectors at ingest time. Only run manually for
songs imported before vector support:
```bash
sf-cli embed-songs --db-path /mnt/wsl/data4tb/static-flow-data/lancedb-music
sf-cli ensure-indexes --db-path /mnt/wsl/data4tb/static-flow-data/lancedb
```
## Error Handling
| Error | Action |
|-------|--------|
| Netease 403/VIP/copyright | Switch to Bilibili Flow E; keep Netease for lyrics |
| Bilibili cookie expired | `ncmdump-cli bili-login` (QR, lasts ~6 months) |
| Netease cookie expired | `ncmdump-cli login <MUSIC_U>` |
| Network timeout | Retry up to 3 times, 5s delay |
| Duplicate ID | Skip unless user confirms overwrite |
| Cover URL missing | Fetch from source API. Never leave empty |
| Cover URL is http:// | Convert to https:// before writing |
| M4A disguised as MP3 | `file <path>` to detect, ffmpeg to convert |
| ffmpeg not found | Install before Bilibili downloads |
## Output Contract
Report after each ingestion:
- `id`, `title`, `artist`, `album`, `album_id`
- `cover_image` (confirm non-empty, https)
- `format` (mp3/flac), `duration_ms`
- `status`: success or error message
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |