Skip to content

K-Dense-AI/scientific-agent-skills / skills/latchbio-integration

latchbio-integration

Verified

Builds, registers, debugs, and operates bioinformatics workflows on Latch using the Python SDK, CLI, Latch Data and Registry, Nextflow, Snakemake, programmatic execution, and Latch MCP. Use when authoring or deploying Latch workflows, configuring resources or interfaces, moving data, integrating Registry, or launching and monitoring runs.

references/nextflow-snakemake.md

View on GitHub
# Nextflow and Snakemake Integration

Latch supports Python, Nextflow, and Snakemake workflows, but their packaging
paths are not interchangeable. Confirm the SDK version and choose one track
before generating files.

## Nextflow: Documented SDK Wrapper Path

### Prerequisites

- A runnable Nextflow pipeline
- A `nextflow_schema.json`
- Containers or a documented execution profile for every process
- A pinned Latch SDK

Install:

```bash
uv pip install "latch==2.77.1"
```

### Generate metadata

```bash
latch generate-metadata nextflow_schema.json --nextflow
```

Current generation creates:

```text
latch_metadata/
├── __init__.py
└── generated.py
```

- `generated.py` is regenerated from the schema. Do not edit it.
- Put persistent custom metadata and flow changes in `latch_metadata/__init__.py`.
- Re-run generation after changing `nextflow_schema.json`.
- Review inferred file, enum, default, and required types before registration.

SDK 2.67.0 changed Nextflow generation to use one generated dataclass containing
all parameters and a generated base flow. Older `parameters.py` examples may no
longer match the generated layout.

### Register

The currently documented wrapper command is:

```bash
latch login
latch register . \
  --nf-script main.nf \
  --nf-execution-profile docker,test
```

Registration generates a Latch workflow wrapper and `latch.config`. The exact
entrypoint location depends on the generation path and SDK version.

Important:

- If a root `Dockerfile` exists, registration uses it.
- Otherwise Latch can generate a Dockerfile under `.latch/`.
- Passing `--nf-script` again can regenerate and overwrite wrapper code.
- After intentionally customizing a generated entrypoint, re-register without
  `--nf-script` to preserve it.
- Keep custom code in a separate module when possible.

### Generate an entrypoint explicitly

SDK 2.77.1 also exposes:

```bash
latch nextflow generate-entrypoint . \
  --nf-script main.nf \
  --execution-profile docker,test \
  --output wf/custom_entrypoint.py
```

This requires a valid `NextflowMetadata` object in the metadata root.

### Experimental Forch-only registration

The CLI also has:

```bash
latch nextflow register . --script-path main.nf
```

The official CLI guide marks this command **experimental** and only applicable
to Forch, Latch's architecture for running Nextflow in the user's own AWS
account. It is not a general alternative to `latch register --nf-script`.
Use it only for a configured Forch/BYOC project with current Latch guidance.

### Nextflow configuration rules

- Define every process container.
- Use profiles for environment-specific settings rather than editing the
  pipeline for Latch.
- Keep Latch's generated `latch.config` in the effective config chain.
- Use the official Latch shared-storage guidance for processes that need a
  common work directory.
- Configure private registries through Latch's supported credentials path.
- Follow the Latch GPU guide for process accelerators; do not translate Python
  task decorators into Nextflow resource syntax.
- Set `NextflowRuntimeResources.storage_gib` for the runtime/shared filesystem,
  not just final outputs.
- Understand storage-retention cost before increasing
  `storage_expiration_hours`.

### Debugging

```bash
latch register --staging .
latch develop .
latch nextflow attach --execution-id <execution-id>
```

Within `latch develop`, the production storage initializer is unavailable.
Follow the official debug-mode guidance: use the local executor, bypass the
initializer in debug mode, use a compatible Latch Nextflow base image, and
reduce process resources.

## Snakemake: Choose a Compatibility Track

The current docs and current stable package expose different Snakemake tracks.
Do not mix them.

### Track A: legacy flags in 2.77.1 source (documentation conflict)

The `2.77.1` wheel still exposes the `snakemake` extra, legacy metadata classes,
`generate-metadata --snakemake`, and `register --snakefile`. However, the
official CLI guide marks the Snakemake metadata and registration flags
deprecated and says metadata generation no longer works for
`latch >= 2.55.0.a6`.

Treat this as an unresolved source/documentation conflict. The commands below
are useful when maintaining a legacy project already known to use this path,
but should not be presented as a supported new-project flow without confirming
with current Latch documentation or support.

Use Python 3.11 for the broadest compatibility with the pinned Snakemake 7.x
dependency:

```bash
uv venv --python 3.11
source .venv/bin/activate
uv pip install "latch[snakemake]==2.77.1"
```

Generate metadata from the workflow config:

```bash
latch generate-metadata config.yaml --snakemake
```

Register:

```bash
latch register . --snakefile Snakefile
```

Relevant options:

```bash
latch register . \
  --snakefile Snakefile \
  --metadata-root latch_metadata \
  --cache-tasks
```

Use the current `SnakemakeMetadata`, `SnakemakeParameter`, `FileMetadata`,
`EnvironmentConfig`, and `DockerMetadata` APIs from `latch.types.metadata`.
Inspect their signatures before generating hand-written metadata.

### Track B: official Snakemake v2 tutorial

The current Snakemake v2 tutorial is a compatibility-specific path. At the time
of this refresh, it explicitly requires:

```bash
uv venv --python 3.11
source .venv/bin/activate
uv pip install "latch==2.62.1a2"
```

It uses imports such as:

```python
from latch.types.metadata.snakemake_v2 import SnakemakeV2Metadata
```

and commands such as:

```bash
latch snakemake generate-entrypoint .
latch dockerfile --snakemake -c environment.yaml . -f
latch register -y .
```

Those v2 metadata modules and the `latch snakemake` command group are not
present in the stable 2.77.1 source tree. The tutorial's generated Dockerfile
also pins the workflow runtime separately to
`latch[snakemake]==2.55.0.a6`.

Therefore:

- Re-check the official tutorial's exact pin before starting.
- Use an isolated environment.
- Preserve and review both the local CLI pin and generated runtime pin.
- Do not upgrade that environment to stable 2.77.1 without a migration plan.
- Do not copy v2 imports into a stable-track project.
- Treat the alpha pin as pre-release software and validate end to end.

### Snakemake resource and environment rules

- Give every rule CPU and memory resources or define safe defaults in
  `profiles/default/config.yaml`.
- Pin Conda and container environments.
- Keep the Latch executor/storage plugin configuration from the chosen track.
- Use `LatchOutputDir` for output destinations exposed in the form.
- Avoid embedding credentials in `environment.yaml`, Dockerfiles, profiles, or
  generated metadata.

## Generated-File Discipline

Before editing a generated file:

1. Find the command and source file that generated it.
2. Determine whether regeneration overwrites the file.
3. Prefer the documented extension file.
4. If customization of generated code is unavoidable, record the generation
   command and stop passing flags that regenerate it.
5. Diff generated output after every SDK upgrade.

## Release Checklist

- SDK and pipeline-engine versions are pinned.
- Schema/config and generated metadata are in sync.
- Every parameter has the expected type and default.
- Containers and package environments are immutable enough to reproduce.
- Small test data succeeds.
- Resume/cache behavior has been exercised.
- Shared storage has enough capacity and an intentional retention period.
- Result paths and execution reports appear in the Console.
- A clean registration from a fresh checkout works.

## Official Sources

- Nextflow tutorial: https://wiki.latch.bio/workflows/sdk/nextflow/tutorial
- Nextflow overview: https://wiki.latch.bio/workflows/sdk/nextflow/overview
- Nextflow dependencies: https://wiki.latch.bio/workflows/sdk/nextflow/dependencies
- Nextflow profiles: https://wiki.latch.bio/workflows/sdk/nextflow/profiles
- Nextflow GPU guide: https://wiki.latch.bio/workflows/sdk/nextflow/gpus
- Nextflow shared storage: https://wiki.latch.bio/workflows/sdk/nextflow/shared-storage
- Snakemake v2 tutorial: https://wiki.latch.bio/workflows/sdk/snakemake-v2/tutorial
- Snakemake v2 overview: https://wiki.latch.bio/workflows/sdk/snakemake-v2/overview
- CLI source in the 2.77.1 release commit: https://github.com/latchbio/latch/blob/b3768e65c6d496868f6e530f11977d857ad85dc7/src/latch_cli/main.py
- Changelog in the 2.77.1 release commit: https://github.com/latchbio/latch/blob/b3768e65c6d496868f6e530f11977d857ad85dc7/CHANGELOG.md
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall