Skip to content

zenstory-ai/video-recap-skills / skills/video-recap

video-recap

Verified

把一段或多集视频做成带画框字幕的中文解说成片或纯原声高光短片。用户提供 .mp4 / .mov / .mkv / .webm,要求做影视解说、短剧二创、加旁白配音、跨集混剪或导出剪映草稿时使用;串联完成看片识台词、定主线与留原声、按台词停顿切镜头、写词检查、TTS 配音与混音加字幕。触发词:视频解说、影视解说、视频旁白、生成解说、视频 recap、video recap、voiceover、narration、recap。

FileKindSize
SKILL.mdskill md18.4 KB
assets/dashboard/app.jsscript16.2 KB
assets/dashboard/index.htmlother867 B
assets/dashboard/styles.cssother22.6 KB
assets/dashboard/tokens.cssother8.9 KB
assets/dashboard/views.jsscript31.4 KB
references/audio-modes.mdmarkdown3.6 KB
references/config-playbook.mdmarkdown8.1 KB
references/creative-editing-playbook.mdmarkdown12.3 KB
references/data-schema.mdmarkdown34.6 KB
references/index-tts.mdmarkdown2.1 KB
references/jianying/LICENSE.duo-videoother1.0 KB
references/jianying/SOURCE.mdmarkdown703 B
references/jianying/empty_draft_meta_info.jsonconfig1.7 KB
references/jianying/empty_jy_material_video.jsonconfig1.8 KB
references/jianying/empty_jy_meta_material_value.jsonconfig358 B
references/jianying/empty_jy_project_info.jsonconfig3.6 KB
references/jianying/empty_jy_segment.jsonconfig1.3 KB
references/jianying/empty_jy_text_styles.jsonconfig440 B
references/jianying/empty_yj_material_audio.jsonconfig865 B
references/jianying/empty_yj_material_text.jsonconfig2.4 KB
references/packaging.mdmarkdown6.8 KB
references/promotional-copy.mdmarkdown4.1 KB
references/prompt-templates.mdmarkdown2.0 KB
references/research-guide.mdmarkdown2.7 KB
references/resource-library.mdmarkdown12.1 KB
references/shot-review.mdmarkdown4.8 KB
scripts/agent_text.pyscript2.7 KB
scripts/artifacts.pyscript972 B
scripts/asr.pyscript12.5 KB
scripts/asr_timing_evidence.pyscript4.2 KB
scripts/assemble.pyscript31.0 KB
scripts/assemble_constants.pyscript894 B
scripts/assembly_contract.pyscript10.3 KB
scripts/assembly_settings.pyscript3.8 KB
scripts/audio_automation.pyscript6.6 KB
scripts/audio_mix.pyscript18.5 KB
scripts/briefing/__init__.pyscript101 B
scripts/briefing/builder.pyscript31.1 KB
scripts/briefing/context.pyscript6.1 KB
scripts/briefing/timeline.pyscript23.0 KB
scripts/codec_peak.pyscript5.1 KB
scripts/cut.pyscript148 B
scripts/cut_cli.pyscript12.4 KB
scripts/cut_contract.pyscript13.3 KB
scripts/cut_qc.pyscript1.9 KB
scripts/cut_render.pyscript13.2 KB
scripts/dashboard/__init__.pyscript159 B
scripts/dashboard/data.pyscript24.6 KB
scripts/dashboard/io.pyscript7.1 KB
scripts/dashboard/runs.pyscript21.8 KB
scripts/dashboard/templates.pyscript9.6 KB
scripts/dashboard_server.pyscript9.3 KB
scripts/detect.pyscript22.2 KB
scripts/doctor.pyexecutablescript12.9 KB
scripts/entry_suggestion.pyscript4.6 KB
scripts/export_jianying.pyscript5.6 KB
scripts/extract.pyscript2.5 KB
scripts/final_qc.pyscript19.0 KB
scripts/frame_grid.pyscript10.6 KB
scripts/jianying/__init__.pyscript49 B
scripts/jianying/builders.pyscript11.3 KB
scripts/jianying/model.pyscript2.7 KB
scripts/jianying/optional.pyscript1012 B
scripts/jianying/schema.pyscript2.5 KB
scripts/jianying/templates.pyscript1.0 KB
scripts/jianying/timeline_contract.pyscript5.8 KB
scripts/jianying/tracks.pyscript2.0 KB
scripts/jianying/writer.pyscript7.8 KB
scripts/lib.pyscript29.6 KB
scripts/library.pyscript31.1 KB
scripts/lint_summary.pyscript5.0 KB
scripts/loudness.pyscript14.7 KB
scripts/materials.pyscript22.1 KB
scripts/media.pyscript18.8 KB
scripts/narration_audio.pyscript13.5 KB
scripts/narration_lint.pyscript24.1 KB
scripts/narrative_selection.pyscript8.9 KB
scripts/packaging.pyscript6.1 KB
scripts/packaging_fonts.pyscript12.0 KB
scripts/packaging_plates.pyscript31.2 KB
scripts/packaging_studio.pyscript34.1 KB
scripts/providers/__init__.pyscript72 B
scripts/providers/fish_audio.pyscript3.1 KB
scripts/providers/index_tts.pyscript8.8 KB
scripts/recap.pyscript183 B
scripts/recap_cli.pyscript6.9 KB
scripts/recap_inspect.pyscript16.5 KB
scripts/recap_runner.pyscript22.0 KB
scripts/recap_runtime.pyscript11.8 KB
scripts/recap_source.pyscript1.8 KB
scripts/recap_stage_qc.pyscript2.7 KB
scripts/recap_timeline.pyscript7.7 KB
scripts/render_preflight.pyscript1.7 KB
scripts/resources/__init__.pyscript200 B
scripts/resources/lock.pyscript6.8 KB
scripts/resources/project_binding.pyscript16.1 KB
scripts/sentence_boundaries.pyscript28.8 KB
scripts/sentence_gate.pyscript8.6 KB
scripts/sfx_mix.pyscript14.6 KB
scripts/shot_review.pyscript18.9 KB
scripts/source_subtitles.pyscript7.2 KB
scripts/speech_ownership.pyscript9.3 KB
scripts/storyboard.pyscript21.8 KB
scripts/subtitles/__init__.pyscript56 B
scripts/subtitles/core.pyscript19.0 KB
scripts/subtitles/render.pyscript8.0 KB
scripts/timeline.pyscript8.2 KB
scripts/timeline_emit.pyscript5.4 KB
scripts/tts_audio.pyscript9.6 KB
scripts/tts_cache.pyscript4.5 KB
scripts/understand.pyscript171 B
scripts/understanding_brief.pyscript6.3 KB
scripts/understanding_cache.pyscript8.0 KB
scripts/understanding_runner.pyscript9.8 KB
scripts/understanding_storyboard.pyscript11.9 KB
scripts/validate.pyscript5.7 KB
scripts/visual_render.pyscript10.1 KB
scripts/vlm.pyscript16.4 KB
scripts/voiceover.pyscript28.0 KB

scripts/voiceover.py

View on GitHub
import argparse
import base64
import json
import os
import re
import tempfile
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path

from agent_text import strip_inline_voice_cues
from providers.fish_audio import fish_speed, synthesize_fish_audio
import providers.index_tts as index_provider
from lib import (
    CONFIG,
    file_identity,
    get_video_duration,
    log,
    mimo_tts_api_call,
    narration_tempo_budget,
    run_cmd,
)
import tts_cache
from tts_audio import (
    _maybe_normalize_tts_wav,
    implausible_tts_duration,
    rejected_take_hint,
    rejected_take_path,
)

SUPPORTED_TTS_ENGINES = {"mimo-tts", "fish-audio", "index-tts"}
SEGMENT_AUDIO_SCHEMA_VERSION = 1
VOICE_REFERENCE_PREP_VERSION = 1


def _parse_rate_offset(rate_str):
    """'+5%' -> 0.05, '-3%' -> -0.03, '+0%' -> 0.0"""
    return float(rate_str.rstrip("%")) / 100.0


def _clean_narration_text(text):
    """清理解说文本中 TTS 不应读出的内容"""
    # 移除 markdown 格式标记
    text = re.sub(r'\*\*(.+?)\*\*', r'\1', text)  # **bold** → bold
    text = re.sub(r'\*(.+?)\*', r'\1', text)       # *italic* → italic
    text = re.sub(r'「(.+?)」', r'\1', text)        # 「quote」 → quote
    text = re.sub(r'」|「|『|』', '', text)
    # 移除方括号舞台指示([climax]、[suspense] 等)及闭集英文语气词,保留普通圆括号正文
    text = strip_inline_voice_cues(text)
    # 规范化省略号和重复标点
    text = re.sub(r'\.{3,}|…{2,}', '……', text)
    text = re.sub(r'……+', '……', text)
    text = re.sub(r'([。!?,;:])\1+', r'\1', text)  # 重复标点 → 单个
    # 移除 emoji
    text = re.sub(r'[\U0001F600-\U0001F64F\U0001F300-\U0001F5FF\U0001F680-\U0001F6FF'
                  r'\U0001F1E0-\U0001F1FF\U00002702-\U000027B0\U0000FE00-\U0000FEFF]', '', text)
    # 清理多余空白
    return re.sub(r'\s+', ' ', text).strip()


def _synthesize_segment(i, seg, narration, tts_dir, engine, prepared=None, voice_ref_b64=None):
    """合成单个 TTS 段(线程安全),支持 resume 跳过已有文件。

    `prepared` is the `_prepare_tts_segment` tuple of a segment synthesize_tts already probed
    as a cache miss; passing it skips a second sidecar read + settings resolution.
    `voice_ref_b64` is the reference audio synthesize_tts transcoded once for this run."""
    if prepared is None:
        prepared = _prepare_tts_segment(i, seg, narration, tts_dir, engine)
        if prepared is None:
            return None
        cached = _reuse_tts_segment_cache(
            i, seg, prepared[1], prepared[4], engine, _parse_rate_offset(prepared[2]))
        if cached:
            return cached
    text, output_wav, rate, pitch, cache_inputs = prepared

    # output_wav may be a hard link into the cache (another block's audio until now): unlink it
    # so the provider writes a new file instead of overwriting that shared audio in place.
    _cleanup_partial_tts_outputs(output_wav)
    provider_receipt = _run_tts_engine(
        engine, text, output_wav, rate=rate, pitch=pitch, emotion=seg.get("emotion"),
        voice_ref_b64=voice_ref_b64, segment_number=i + 1,
    )

    dur = get_video_duration(output_wav)
    rate_offset = _parse_rate_offset(rate)
    _check_segment_window(i, seg, dur, rate_offset)

    norm_meta = _maybe_normalize_tts_wav(output_wav)
    if norm_meta:
        dur = get_video_duration(output_wav)
    _write_tts_segment_cache(output_wav, cache_inputs, text, dur, norm_meta, provider_receipt)
    return _build_tts_segment_result(
        i, seg, text, output_wav, dur, rate_offset, norm_meta, provider_receipt)


def _check_segment_window(index, seg, duration, rate_offset):
    """Log a block that overflows its window; the authored text is never shortened here."""
    seg_slot = seg["end"] - seg["start"]
    seg_pause = seg.get("pause_after_ms", CONFIG["breath_ms"]) / 1000
    available = max(0.5, seg_slot - seg_pause)
    raw_budget = available * narration_tempo_budget(rate_offset)["max_raw_duration_factor"]
    if duration > raw_budget:
        # Never shorten authored text here: a truncated segment always blocked later as
        # `truncated_speech`. assemble either fits it with bounded tempo or blocks it as
        # `no_safe_fit` before the video encode.
        log(
            f"  段 {index+1}: 超出预算 {duration:.1f}s > {raw_budget:.1f}s,保留原稿,"
            "交由 assemble 有界提速或在渲染前阻断"
        )


def _build_tts_segment_result(index, seg, text, output_wav, duration, rate_offset,
                              norm_meta=None, provider_receipt=None):
    budget = narration_tempo_budget(rate_offset)
    authored_text = _clean_narration_text(seg["narration"])
    # voiceover never shortens text; still flag any spoken/authored mismatch (e.g. a reused sidecar).
    resolved_truncated = text != authored_text
    result = {
        "segment_audio_schema_version": SEGMENT_AUDIO_SCHEMA_VERSION,
        "index": index,
        "start": seg["start"],
        "end": seg["end"],
        "narration": authored_text,
        "spoken_text": text,
        "truncated": resolved_truncated,
        "truncate_reason": "sentence_boundary" if resolved_truncated else "none",
        "fit_status": "pending_assembly",
        "audio_path": str(output_wav),
        "audio_duration": duration,
        "placed_audio_duration": None,
        "actual_place_start": None,
        "actual_place_end": None,
        "global_narration_speed": budget["global_narration_speed"],
        "segment_tempo_factor": 1.0,
        "effective_tempo": budget["global_narration_speed"] * budget["tts_rate_factor"],
        "rms_dbfs_before": norm_meta["rms_dbfs_before"] if norm_meta else None,
        "rms_dbfs_after": norm_meta["rms_dbfs_after"] if norm_meta else None,
        "peak_after": norm_meta["peak_after"] if norm_meta else None,
        "tts_rate_offset": rate_offset,
        "pause_after_ms": seg.get("pause_after_ms", CONFIG["breath_ms"]),
        "overlaps_speech": seg.get("overlaps_speech", True),
        "provider_receipt": provider_receipt,
    }
    for optional_key in ("source_start", "source_end", "source_clip_id", "emotion"):
        if optional_key in seg:
            result[optional_key] = seg[optional_key]
    return result


def _tts_failure_record(index, seg, error):
    """Build a user-visible failure record for partial TTS output."""
    return {
        "index": index,
        "start": seg["start"],
        "end": seg["end"],
        "text": _clean_narration_text(seg["narration"]),
        "error": str(error),
    }


def _voice_record(engine):
    """Which voice this run actually used: provider, model, voice id or reference audio."""
    settings = tts_settings_payload(engine)
    if engine == "fish-audio":
        return {"provider": engine, "model": settings["fish_tts_model"],
                "voice_id": settings["fish_tts_reference_id"], "reference": None}
    if engine == "index-tts":
        return {"provider": engine, "model": None,
                "voice_id": settings["index_tts_voice"], "reference": None}
    reference = settings.get("voice_ref_identity")
    return {"provider": engine, "model": settings["mimo_tts_model"],
            "voice_id": None if reference else settings["mimo_tts_voice"],
            "reference": reference}


def _build_tts_meta(segments, engine, narration_name, failures):
    """Stable tts_meta.json payload, including partial-failure visibility."""
    return {
        "segments": segments,
        "engine": engine,
        "voice": _voice_record(engine),
        "narration": narration_name,
        "partial": bool(failures),
        "failures": failures,
    }


def synthesize_tts(narration, work_dir):
    """合成解说音频(并行)。Returns (segments, engine, failures)."""
    voice_ref = CONFIG["voice_ref"]
    tts_dir = work_dir / "tts_segments"
    tts_dir.mkdir(exist_ok=True)

    if not narration:
        raise RuntimeError("narration.json 没有可配音的解说段,已中止以避免生成无解说视频")

    cache_engine = _configured_tts_engine_for_cache()
    if cache_engine in {"fish-audio", "index-tts"} and voice_ref:
        if cache_engine == "fish-audio":
            raise RuntimeError("Fish Audio 不接受本地 VOICE_REF/--voice-ref;请改用 FISH_TTS_REFERENCE_ID")
        raise RuntimeError("index-tts 不支持本地 VOICE_REF/--voice-ref 克隆")
    # A fully cached narration needs no credential: probe every segment's cache once here;
    # hits are final and only misses reach the workers (with their prepared inputs).
    segments = []
    misses = []
    failures = []
    for i, seg in enumerate(narration):
        prepared = _prepare_tts_segment(i, seg, narration, tts_dir, cache_engine)
        if prepared is None:
            continue
        cached = _reuse_tts_segment_cache(
            i, seg, prepared[1], prepared[4], cache_engine, _parse_rate_offset(prepared[2]))
        if cached:
            segments.append(cached)
        else:
            misses.append((i, seg, prepared))
    if not segments and not misses and not failures:
        raise RuntimeError("narration.json 没有可配音的有效文本,已中止以避免生成无解说视频")
    if misses:
        engine = _synthesize_misses(misses, narration, tts_dir, segments, failures)
    else:
        engine = cache_engine
        log(f"TTS 引擎: {cache_engine} (cache)")
    return _finish_tts(narration, segments, engine, failures)


def _synthesize_misses(misses, narration, tts_dir, segments, failures):
    """Synthesize the cache misses in parallel into `segments` / `failures`; returns the engine."""
    engine = resolve_tts_engine()
    voice_ref = CONFIG["voice_ref"]

    # Transcode the reference once per run, and only now: a fully cached rerun never runs
    # ffmpeg. The misses' cache keys hold the identity probed above, so a reference edited
    # meanwhile would label new audio with the old identity; fail instead.
    voice_ref_b64 = None
    if voice_ref and engine == "mimo-tts":
        probed_identity = misses[0][2][4]["settings"]["voice_ref_identity"]  # [4] = cache inputs
        voice_ref_b64 = _prepare_voice_reference(voice_ref)
        if _voice_reference_signature(voice_ref) != probed_identity:
            raise RuntimeError(f"参考音频在配音期间被修改: {voice_ref};请重新运行")

    log(f"TTS 引擎: {engine}")

    max_workers = min(len(misses), CONFIG["tts_workers"])

    with ThreadPoolExecutor(max_workers=max_workers) as executor:
        futures = {
            executor.submit(
                _synthesize_segment, i, seg, narration, tts_dir, engine, prepared,
                voice_ref_b64=voice_ref_b64,
            ): i
            for i, seg, prepared in misses
        }
        for future in as_completed(futures):
            try:
                result = future.result()
            except Exception as e:
                i = futures[future]
                failures.append(_tts_failure_record(i, narration[i], e))
                log(f"  TTS 段 {i+1} 失败: {e}")
                continue
            if result:
                segments.append(result)
                log(f"  段 {result['index']+1}: {result['audio_duration']:.1f}s - {result['narration'][:25]}...")
    return engine


def _finish_tts(narration, segments, engine, failures):
    """Apply the failure policy to one run's results; returns (segments, engine, failures)."""
    segments.sort(key=lambda x: x["index"])
    failures.sort(key=lambda x: x["index"])
    if failures and not CONFIG["allow_partial_tts"]:
        sample = "; ".join(f"段 {f['index']+1}: {f['error']}" for f in failures[:3])
        raise RuntimeError(
            f"TTS 失败 {len(failures)}/{len(narration)} 段,已中止以避免生成缺解说的视频。"
            f"示例: {sample}。如确需继续,可传 --allow-partial-tts。"
        )
    if failures:
        missing = "、".join(f"段 {f['index'] + 1}" for f in failures[:8])
        more = "…" if len(failures) > 8 else ""
        log(
            f"警告: TTS 部分失败 {len(failures)}/{len(narration)} 段({missing}{more}),"
            "成片可预览但不建议直接发布;详见 tts_meta.json failures"
        )
    if not segments:
        raise RuntimeError("TTS 没有生成任何有效解说音频,已中止以避免生成无解说视频")
    return segments, engine, failures


def _run_tts_engine(engine, text, output_wav, rate="+0%", pitch="+0Hz", emotion=None,
                    voice_ref_b64=None, segment_number=None):
    """Run one TTS engine with retry and remove partial files after failures.

    `engine` comes from resolve_tts_engine; index-tts controls were already forced to the
    provider defaults by _prepare_tts_segment. A take rejected as implausibly long is kept as
    `<name>.rejected.wav` (the latest one) so the user can listen before relaxing the bound."""
    retries = CONFIG["tts_retries"]
    last_error = None
    rejected = rejected_take_path(output_wav)
    rejected.unlink(missing_ok=True)
    label = f"段 {segment_number} " if segment_number is not None else ""

    for attempt in range(1, retries + 1):
        try:
            _cleanup_partial_tts_outputs(output_wav)
            receipt = None
            if engine == "mimo-tts":
                _tts_mimo(text, output_wav, rate=rate, pitch=pitch, emotion=emotion,
                          voice_ref_b64=voice_ref_b64)
            elif engine == "fish-audio":
                synthesize_fish_audio(text, output_wav, rate=rate)
            else:
                receipt = index_provider.synthesize_configured(text, output_wav, CONFIG)
            duration = get_video_duration(output_wav)
            if duration <= 0:
                raise RuntimeError(f"{engine} 输出音频时长无效")
            # A hallucinated reading (the text plus invented speech) is a failed attempt: it is
            # retried, never cached, and never reaches assemble as an over-budget block.
            implausible = implausible_tts_duration(text, duration)
            if implausible:
                os.replace(output_wav, rejected)
                raise RuntimeError(implausible)
            rejected.unlink(missing_ok=True)
            return receipt
        except Exception as exc:
            last_error = exc
            _cleanup_partial_tts_outputs(output_wav)
            if attempt < retries:
                wait = min(2 ** (attempt - 1), 8)
                log(f"  {label}TTS 重试 {attempt+1}/{retries}: {exc},等待 {wait}s")
                time.sleep(wait)

    hint = rejected_take_hint(rejected)
    raise RuntimeError(f"{engine} 合成失败: {last_error}{hint}") from last_error


def _cleanup_partial_tts_outputs(output_wav):
    """Remove stale partial media files before/after a failed TTS attempt."""
    wav_path = Path(output_wav)
    for path in (
        wav_path,
        wav_path.with_suffix(".mp3"),
        Path(str(wav_path) + ".part"),
    ):
        path.unlink(missing_ok=True)


def _prepare_tts_segment(index, seg, narration, tts_dir, engine):
    del narration
    text = _clean_narration_text(seg["narration"])
    if not text:
        return None
    output_wav = tts_dir / f"narr_{index:03d}.wav"
    if engine == "index-tts":
        rate, pitch = index_provider.default_controls(seg)
    else:
        rate, pitch = "+0%", "+0Hz"
    cache_inputs = _tts_segment_cache_inputs(engine, seg, text, rate, pitch)
    return text, output_wav, rate, pitch, cache_inputs


def _reuse_tts_segment_cache(index, seg, output_wav, cache_inputs, engine, rate_offset):
    """A result from the content-addressed cache, with `output_wav` pointing at its audio.

    `rate_offset` is this block's own nominal rate. The key holds only what the provider
    receives, so a take synthesized at another nominal rate that produced the same request
    is reused, and reports this block's rate exactly as a fresh synthesis would."""
    cached = tts_cache.load(output_wav.parent, cache_inputs)
    if cached is None:
        return None
    if engine == "index-tts" and not index_provider.valid_cached_receipt(cached, CONFIG):
        return None
    implausible = implausible_tts_duration(cached["spoken_text"], cached["audio_duration"])
    if implausible:
        # Written before this bound existed: re-synthesize instead of blocking every rerun.
        log(f"  段 {index+1}: 不复用缓存,重新合成:{implausible}")
        return None
    # The window is not part of the key (it never changes the audio), so a block moved into a
    # shorter window is re-checked here instead of re-synthesized.
    _check_segment_window(index, seg, cached["audio_duration"], rate_offset)
    tts_cache.materialize(output_wav.parent, cache_inputs, output_wav)
    # The sidecar's audio identity (size, mtime_ns) still matches the WAV that produced
    # `audio_duration`; re-probing would be one ffprobe process per segment on every rerun.
    log(f"  段 {index+1}: 复用已有 ({cached['audio_duration']:.1f}s)")
    return _build_tts_segment_result(
        index,
        seg,
        cached["spoken_text"],
        output_wav,
        cached["audio_duration"],
        rate_offset,
        cached["normalization"],
        cached.get("provider_receipt"),
    )


def _provider_prosody_request(engine, rate, pitch, emotion):
    """The prosody part of the request the provider actually receives for one block.

    MiMo gets a natural-language instruction in which rate offsets only change the wording
    at >= +6% or <= -3% (pitch only as zero/non-zero); Fish Audio gets a numeric speed and
    ignores pitch and emotion; index-tts takes no per-block controls."""
    if engine == "mimo-tts":
        return {"instruction": _mimo_tts_style_instruction(rate, pitch, emotion)}
    if engine == "fish-audio":
        return {"speed": fish_speed(rate)}
    return {}


def _tts_segment_cache_inputs(engine, seg, source_text, rate, pitch):
    """The exact inputs that make cached audio safe to reuse (compared by equality).

    Only what changes the audio: text, the prosody request the provider receives, and
    provider/voice settings. The block's position and window are left out, so deleting or
    inserting a block reuses every other block's audio; a block whose position changes its
    nominal rate is re-synthesized only if the provider would receive a different request."""
    return {
        "engine": engine,
        "source_text": source_text,
        "provider_request": _provider_prosody_request(engine, rate, pitch, seg.get("emotion")),
        "settings": tts_settings_payload(engine),
    }


def _write_tts_segment_cache(output_wav, cache_inputs, spoken_text, duration,
                             norm_meta=None, provider_receipt=None):
    """Store the finished block WAV under its non-secret synthesis inputs for reuse."""
    tts_cache.store(output_wav, cache_inputs, {
        "spoken_text": spoken_text,
        "audio_duration": duration,
        "normalization": norm_meta or None,
        "provider_receipt": provider_receipt,
    })


def _configured_tts_engine_for_cache():
    """Resolve provider intent without requiring a live credential for cache probes."""
    provider = CONFIG["tts_provider"]
    if provider == "auto":
        if CONFIG["mimo_tts_api_key"] or not CONFIG["fish_api_key"]:
            return "mimo-tts"
        return "fish-audio"
    if provider not in SUPPORTED_TTS_ENGINES:
        raise RuntimeError(
            "TTS_PROVIDER/--tts-provider 必须是 auto、mimo-tts、fish-audio 或 index-tts"
        )
    return provider


def resolve_tts_engine():
    """Resolve the selected MiMo or Fish Audio TTS engine and require its credential."""
    engine = _configured_tts_engine_for_cache()
    if engine == "index-tts":
        return engine
    if engine == "fish-audio":
        if CONFIG["fish_api_key"]:
            return engine
        raise RuntimeError("没有可用的 TTS 引擎:请设置 FISH_API_KEY(Fish Audio 需要)。")
    if CONFIG["mimo_tts_api_key"]:
        return engine
    raise RuntimeError(
        f"没有可用的 TTS 引擎:请设置 {CONFIG['mimo_tts_env_var']}(MiMo TTS 需要)。"
    )


def tts_settings_payload(engine):
    """Return non-secret TTS settings that materially affect generated audio."""
    settings = {
        "engine": engine,
        "narration_speed": CONFIG["narration_speed"],
        "narration_cumulative_tempo_max": CONFIG["narration_cumulative_tempo_max"],
        "narration_cumulative_tempo_hard_max": CONFIG["narration_cumulative_tempo_hard_max"],
        "tts_segment_tempo_max": CONFIG["tts_segment_tempo_max"],
        "tts_segment_normalize": CONFIG["tts_segment_normalize"],
        "tts_segment_target_rms_dbfs": CONFIG["tts_segment_target_rms_dbfs"],
        "tts_segment_peak_limit": CONFIG["tts_segment_peak_limit"],
    }
    if engine == "index-tts":
        settings.update(index_provider.cache_settings(CONFIG))
    elif engine == "fish-audio":
        settings.update(
            {
                "fish_tts_api_url": CONFIG["fish_tts_api_url"],
                "fish_tts_model": CONFIG["fish_tts_model"],
                "fish_tts_reference_id": CONFIG["fish_tts_reference_id"],
            }
        )
    else:
        settings.update(
            {
                "mimo_tts_api_url": CONFIG["mimo_tts_api_url"],
                "mimo_tts_model": CONFIG["mimo_tts_model"],
                "mimo_tts_voice": CONFIG["mimo_tts_voice"],
                "mimo_tts_style": CONFIG["mimo_tts_style"],
            }
        )
    voice_ref = CONFIG["voice_ref"]
    if voice_ref and engine == "mimo-tts":
        ref_path = Path(voice_ref).expanduser()
        settings.pop("mimo_tts_voice", None)  # ignored by the voiceclone API
        settings["voice_ref_identity"] = _voice_reference_signature(ref_path)
        settings["voice_ref_preparation"] = (
            f"pcm_s16le:24000hz:mono:30s:v{VOICE_REFERENCE_PREP_VERSION}"
        )
        settings["mimo_tts_model"] = "mimo-v2.5-tts-voiceclone"
    return settings


def _mimo_tts_style_instruction(rate="+0%", pitch="+0Hz", emotion=None):
    style = CONFIG["mimo_tts_style"]
    if emotion:
        tone = f"用「{emotion}」的情绪和语气演绎这句解说,代入感强、有起伏,不要平铺直叙。"
    else:
        tone = "语气有感染力、有起伏,像在给观众讲故事,不要平淡机械。"
    rate_offset = _parse_rate_offset(rate)
    if rate_offset >= 0.06:
        speed = "语速略快,但吐字保持清楚。"
    elif rate_offset <= -0.03:
        speed = "语速略慢,适当停顿,保留收束感。"
    else:
        speed = "语速中等,节奏稳定。"
    pitch_hint = "疑问句或情绪抬升处可自然微升调。" if pitch != "+0Hz" else "音调自然。"
    return f"{style} {tone} {speed} {pitch_hint}"


def _prepare_voice_reference(ref_path):
    """Normalize arbitrary reference audio to MiMo voiceclone's 24 kHz mono WAV."""
    ref = Path(ref_path).expanduser()
    if not ref.is_file():
        raise FileNotFoundError(f"参考音频不存在或不是文件: {ref}")
    with tempfile.TemporaryDirectory(prefix="video-recap-voice-ref-") as temp_dir:
        normalized = Path(temp_dir) / "voice_ref.wav"
        result = run_cmd([
            "ffmpeg", "-y", "-i", str(ref), "-vn", "-ar", "24000", "-ac", "1",
            "-t", "30", "-acodec", "pcm_s16le", str(normalized),
        ])
        if result.returncode != 0 or not normalized.is_file() or normalized.stat().st_size <= 44:
            raise RuntimeError(f"参考音频转码失败: {result.stderr.strip() or ref}")
        return base64.b64encode(normalized.read_bytes()).decode("ascii")


def _voice_reference_signature(ref_path):
    """Identity of the reference source: resolved path plus size and mtime_ns."""
    ref = Path(ref_path).expanduser().resolve()
    return {"path": str(ref), **file_identity(ref)}


def _tts_mimo(text, output_path, rate="+0%", pitch="+0Hz", emotion=None, voice_ref_b64=None):
    """使用 Xiaomi MiMo-V2.5-TTS 合成,按需用参考音频克隆音色。

    MiMo-v2.5-tts 是 instruct-TTS:user 消息里的自然语言指令控制整句的情绪/语气/语速。
    每段 narration 的 `emotion` 标签即写进该指令,让解说有起伏、不机械。
    `voice_ref_b64` is the run's prepared reference; a direct call transcodes the live source."""
    voice_ref = CONFIG["voice_ref"]
    if voice_ref and voice_ref_b64 is None:
        voice_ref_b64 = _prepare_voice_reference(voice_ref)
    payload = {
        "model": "mimo-v2.5-tts-voiceclone" if voice_ref else CONFIG["mimo_tts_model"],
        "messages": [
            {"role": "user", "content": _mimo_tts_style_instruction(rate, pitch, emotion)},
            {"role": "assistant", "content": text},
        ],
        "audio": {
            "format": "wav",
            "voice": (
                f"data:audio/wav;base64,{voice_ref_b64}"
                if voice_ref else CONFIG["mimo_tts_voice"]
            ),
        },
    }
    resp = mimo_tts_api_call(payload)
    try:
        audio_data = resp["choices"][0]["message"]["audio"]["data"]
    except (KeyError, IndexError, TypeError) as exc:
        raise RuntimeError("MiMo-TTS 响应缺少 audio.data") from exc
    try:
        output_path.write_bytes(base64.b64decode(audio_data))
    except (TypeError, ValueError) as exc:
        raise RuntimeError("MiMo-TTS 返回的 audio.data 不是有效 base64") from exc


def main():
    ap = argparse.ArgumentParser(
        description="voiceover.py: synthesize narration audio segments from narration.json.")
    ap.add_argument("--work-dir", required=True)
    ap.add_argument("--narration", default=None,
                    help="narration json (default: <work-dir>/narration.json)")
    ap.add_argument("--mimo-voice", default=None, help="MiMo TTS voice name")
    ap.add_argument(
        "--tts-provider",
        default=(os.environ.get("TTS_PROVIDER") or "auto").strip().lower(),
        choices=["auto", "mimo-tts", "fish-audio", "index-tts"],
        help="TTS provider (default: MiMo when configured, otherwise Fish Audio)",
    )
    ap.add_argument("--voice-ref", default=None,
                    help="reference audio (wav/mp3/etc.) for mimo-v2.5-tts-voiceclone")
    ap.add_argument("--allow-partial-tts", action="store_true",
                    help="allow output when some narration segments fail TTS")
    args = ap.parse_args()
    work_dir = Path(args.work_dir)
    CONFIG["tts_provider"] = args.tts_provider
    index_provider.load_private_config(CONFIG, os.environ)
    CONFIG["voice_ref"] = (
        args.voice_ref if args.voice_ref is not None else os.environ.get("VOICE_REF", "").strip()
    )
    if args.mimo_voice:
        CONFIG["mimo_tts_voice"] = args.mimo_voice
    if args.mimo_voice and CONFIG["voice_ref"]:
        ap.error("--mimo-voice and --voice-ref are mutually exclusive")
    if args.mimo_voice and args.tts_provider in {"fish-audio", "index-tts"}:
        ap.error("--mimo-voice is only supported by the MiMo TTS provider")
    # A local --voice-ref with fish-audio/index-tts is rejected by synthesize_tts, which also
    # transcodes the reference lazily: once per run, and never for a fully cached rerun.
    if args.allow_partial_tts:
        CONFIG["allow_partial_tts"] = True
    if args.narration:
        narration_path = Path(args.narration)
    else:
        # Cut mode is cut-first/narrate-second: narration.json is already on the output
        # timeline, so the default needs no remapping.
        narration_path = work_dir / "narration.json"
    narration = json.loads(narration_path.read_text(encoding="utf-8"))
    tts_segments, engine_used, failures = synthesize_tts(narration, work_dir)
    meta = _build_tts_meta(tts_segments, engine_used, narration_path.name, failures)
    tts_cache.write_json_atomically(work_dir / "tts_meta.json", meta)
    if failures:
        log(f"配音完成但缺 {len(failures)} 段:成片可预览但不建议直接发布")
    log(f"配音完成: {len(tts_segments)} 段, 引擎 {engine_used}")
    print(json.dumps({"status": "voiced", "segments": len(tts_segments), "engine": engine_used,
                      "partial": bool(failures), "failures": len(failures),
                      "tts_meta": str(work_dir / "tts_meta.json")}, ensure_ascii=False))


if __name__ == "__main__":
    main()
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 196kUpdated Claude CodeCodexVerifiedInstall